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What  Really  Matters  in  Defense  Acquisition 

Frank  Kendall 


My  first  inclination  for  this  issue's  arti¬ 
cle  was  to  discuss  the  newly  released 
DoDI  5000.02.  We  recently  imple¬ 
mented  this  new  acquisition  policy 
document  as  interim  guidance.  I  pro¬ 
vided  a  cover  letter  explaining  why  I  had  done  a  new 
version  and  outlined  some  of  the  features  of  this 
edition.  I  do  recommend  that  you  look  at  both  the 
cover  letter  and  the  new  document,  but  on  reflection 
I  decided  to  write  about  something  else  for  this  issue. 
An  enormous  amount  of  time  and  energy  goes  into 
designing  our  processes  and  implementing  them, 
but  at  the  end  of  the  day  it  isn't  those  processes  or 
policy  documents  like  5000.02  that  really  drive  our 
results.  What  really  matters  in  defense  acquisition 
is  our  people  and  their  professionalism  and  leader¬ 
ship— so  I  thought  I  would  start  the  new  year  by  writ¬ 
ing  about  that. 

This  past  year  we've  gone  through  a  lot,  and  all  of  our  ac¬ 
quisition  professionals  have  been  asked  to  put  up  with  more 
than  any  workforce  should  have  to  endure.  We've  had  con¬ 
tinuing  budget  turmoil  and  uncertainty,  furloughs,  continuing 
resolutions,  late-breaking  sequestration,  and  most  recently 
a  government  shutdown.  We're  also  living  under  pay  freezes 
and  the  prospect  of  further  budget  reductions  and  staff  re¬ 
ductions.  I  want  to  thank  the  whole  workforce  for  the  way 
you  have  all  coped  with  these  challenges.  While  other  senior 
leaders  and  I  have  been  asking  you  to  improve  our  productiv¬ 
ity  and  achieve  ever  greater  results  for  our  warfighters  and 
the  taxpayer,  you've  also  had  to  work  in  very  challenging  cir¬ 
cumstances.  You've  come  through,  and  it  has  inspired  me  and 
your  other  senior  leaders  to  see  the  way  you've  dealt  with  all 
these  challenges  in  stride.  Thank  you.  Thank  you  personally, 
but  also  on  behalf  of  the  Secretary  and  all  the  senior  leaders 
in  the  Department.  Thank  you  also  for  our  soldiers,  sailors, 
airmen  and  marines  who  benefit  from  your  great  work  as  they 
put  themselves  at  risk  for  our  country. 
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Recently,  I  joined  Dr.  Carter  in  one  of  his  last  official  acts 
as  Deputy  Secretary  in  presenting  the  Packard  Awards  to 
this  year's  recipients.  As  I  write  this,  I'm  looking  forward  to 
going  out  to  the  Defense  Acquisition  University  to  present 
the  USD(AT&L)  awards  for  professionalism  and  developing 
the  work  force  to  some  of  our  outstanding  performers.  I'm 
sorry  that  we  can't  recognize  more  of  our  exceptional  per¬ 
formers— there  are  so  many  of  you,  and  you  all  deserve  to 
be  recognized  for  what  you  do.  During  the  last  few  weeks,  I 
also  have  had  occasion  to  note  the  departure  of  some  of  our 
most  capable  people  who  are  retiring  or  will  soon  retire  from 
government  service.  We  lose  a  lot  of  terrific  people  every  year 
of  course,  and  these  individuals  are  just  examples  of  the  many 
fine  professionals  working  in  defense  acquisition,  technology 
and  logistics.  I  decided  that  for  this  article  I  would  note  the 
contributions  of  some  of  these  people  with  whom  over  the  last 
few  years  I've  had  the  chance  to  work.  They  are  just  examples, 
but  they  are  especially  powerful  examples  of  what  one  can 
accomplish  during  a  career  in  defense  acquisition. 

I'll  start  with  Charlie  Williams,  the  recently  retired  Director  of 
the  Defense  Contract  Management  Agency  (DCMA).  Charlie 
led  DCMA  for  the  past  several  years.  He  started  federal  ser¬ 
vice  in  1982  in  Air  Logistics  Command  in  a  Mid-Level  Manage¬ 
ment  Training  Program.  Charlie  then  rose  through  a  series  of 
contracting,  program  analysis  and  contract  management  posi¬ 
tions  with  the  Air  Force  both  in  the  field  and  at  Air  Force  Head¬ 
quarters.  He  became  Air  Force  Deputy  Assistant  Secretary 
for  Contracting  before  taking  the  reins  at  DCMA.  At  DCMA, 
Charlie  led  the  rebuilding  of  the  organization  after  severe  re¬ 
ductions  in  the  1990s.  He  kept  his  team  together  during  the 
Base  Realignment  and  Closure  move  from  D.C.  to  Richmond, 
and  he  led  the  effort  to  ensure  that  our  contracts  in  support 
of  operations  in  Afghanistan  and  Iraq  were  executed  properly. 

Next  I'll  mention  MajGen  Tim  Crosby,  the  soon-to-retire  Army 
Program  Executive  Officer  (PEO)  for  Aviation.  Tim  has  led 
Army  aviation  programs  since  2008.  He  was  commissioned 
after  graduating  from  the  Citadel  and  started  out  as  a  field 
artillery  officer.  He  moved  quickly  into  aviation  as  a  pilot  be¬ 
fore  following  his  interest  in  research  and  development  and 
flight  testing.  In  acquisition,  he  worked  in  logistics,  training  and 
simulation,  and  test  and  evaluation  before  becoming  a  Product 
Manager,  first  for  the  CH-47  F  and  later  Program  Manager  for 
the  Army's  Armed  Scout.  His  long  tenure  at  PEO  Aviation  is 
marked  by  strong  leadership  in  support  of  our  deployed  forces 
and  in  building  the  capability  of  the  Afghan  Air  Force.  Tim 
embraced  the  Better  Buying  Power  principles  and  was  imple¬ 
menting  them  well  before  Dr.  Carter  and  I  gave  them  a  name. 

Rear  Admiral  Jim  Murdoch  retired  recently  after  serving  as 
the  Navy's  first  PEO  for  Littoral  Combat  Ships  (LCS).  Jim  en- 
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tered  the  Navy  with  an  ROTC  commission  after  graduating 
from  Rensselaer  Polytechnic  Institute  in  mechanical  engineer¬ 
ing.  He  moved  between  surface  combatant  assignments  and 
acquisition  positions.  His  acquisition  assignments  included 
program  management  for  surface  weapons  and  launchers 
and  responsibility  for  integrated  warfare  systems  as  well  as 
program  manager  for  the  Littoral  Combat  Ships.  In  2011,  Jim 
was  handpicked  by  Sean  Stackley  to  lead  the  new  Program 
Executive  Office  for  LCS  sea-frames  and  mission  modules.  He 
stabilized  and  fully  integrated  one  of  the  Navy's  most  complex 
acquisition  endeavors. 

Finally  Scott  Correll,  our  retiring  Air  Force  PEO  for  Space 
Launch,  also  started  his  career  as  an  intern.  From  the  Pacer 
Intern  Contracting  Program  at  Robbins  Air  Force  Base,  where 
he  began  as  a  cost  analyst  and  contract  negotiator  on  the  F-4 
and  F-15,  Scott  rose  through  the  contracting,  supply  chain 
management  and  program  management  fields.  Scott's  di¬ 
verse  positions  include  leadership  positions  at  Military  Sea¬ 
lift  Command  and  TRANSCOM.  I  was  able  to  take  Scott  in  to 
meet  Secretary  Hagel  recently  so  the  Secretary  could  thank 
him  personally  for  saving  the  Department  billions  of  dollars  in 
space  launch  costs— quite  an  achievement  for  our  taxpayers 
and  warfighters. 

The  people  I  mention  above  have  accomplished  a  great  deal 
for  their  country  during  their  careers.  They've  also  had  the  op¬ 
portunity  to  do  exciting  and  fulfilling  work.  People  who  achieve 
this  sort  of  success  over  their  careers  are  what  give  us  the  best 
equipped  military  in  the  world.  All  of  these  people  have  a  lot 
to  be  proud  of.  All  of  you  have  a  lot  to  be  proud  of.  I'm  looking 
forward  to  2014  with  the  hope  that  things  will  improve— and 
there  are  some  signs  that  they  will.  But  mostly  I'm  just  look¬ 
ing  forward  to  another  year  of  working  with  this  terrific  team. 
Thank  you  again  for  all  that  you  do.  & 
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Mini-Warriors 


n  1959,  renowned  physicist  Dr.  Richard  Feynman 
proposed  the  idea  of  very  tiny  machines  that  could 
perform  micro-level  tasks  with  macro-level  effects. 
Today  such  machines  exist,  measuring  only  millime¬ 
ters,  with  gears  and  levers  smaller  than  dust  mites. 
In  smart  phones,  they  change  display  orientations  as 
you  move  the  phone,  and  in  cars,  they  collect  speed, 
acceleration  and  steering  data,  sending  that  data  to  an 
in-vehicle  network. 

These  "microelectromechanical  systems,"  or  MEMS,  are  increasingly  found  in  systems 
we  use  daily,  replacing  larger,  heavier,  and  less  reliable  components  at  lower  costs. 
They  are  freeing  up  space  in  systems,  enabling  performance  improvements  and  added 
functions,  and  they  are  performing  previously  unimaginable  tasks  at  the  micro  level. 
Like  their  significant  transformation  of  commercial  systems,  MEMS  have  the  potential 
to  transform  munitions  across  defense  systems,  as  found  in  MEMS  development  at  the 
Naval  Surface  Warfare  Center  Indian  Head  Explosive  Ordnance  Disposal  Technology 
Division  (Indian  Head  EODTD). 

Revolution  in  Commercial  Products 

"MEMS  promises  to  revolutionize  every  product  category,"  wrote  manufacturing  expert 
Dr.  Xuan  F.  Zha  of  the  National  Institute  of  Standards  and  Technology.  That  revolution 
was  indicated  by  the  integration  of  the  first  commercially  made  MEMS  sensor  into  au¬ 
tomotive  airbags  in  1993.  Measuring  less  than  1  cm2,  this  MEMS  detected  crashes  and 
activated  air  bags,  replacing  sensors  that  were  orders  of  magnitude  larger. 

MEMS,  which  vary  in  design  and  function,  are  produced  by  a  process  associated  with 
making  integrated  circuits.  Using  micromachining,  materials  are  deposited,  molded  and 
etched  on  silicon,  which  is  harder  than  most  metals  and  has  semiconductor  properties. 
The  result  is  a  micro  system,  with  tiny  moving  parts  and  microelectronic  circuits  that 
sense  and  act. 

The  benefits  of  MEMS  have  been  increasingly  recognized.  Not  only  are  they  smaller 
and  lighter  than  the  devices  they  replace,  but  they  are  cheaper  because  the  cost  of 
manufacturing  and  materials  is  smaller.  At  just  $5  apiece,  the  first  commercial  MEMS 
reduced  air-bag  systems'  costs  from  roughly  $500  to  $100.  It  also  proved  highly  reli¬ 
able.  A  decade  later,  the  maker  of  that  MEMS  accelerometer  had  sold  more  than  100 
million  units  for  air  bags  and  other  systems,  reporting  "less  than  one  failure  per  billion 
hours  of  operation." 

The  integration  of  MEMS  devices  for  other  uses  also  increased  system  functionality.  In 
less  than  a  decade,  the  BMW  740i  had  more  than  70  MEMS  devices,  enabling  anti-lock 


Jean  and  Cochran  are  mechanical  engineers  and  Pines  is  a  materials  scientist  in  the  Fuze  and  Initiation 
Systems  Branch  at  the  Naval  Surface  Warfare  Center  Indian  Head  Explosive  Ordnance  Disposal  Technol¬ 
ogy  Division. 
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braking,  active  suspension,  appliance  and  navigation  control, 
vibration  monitoring,  fuel  sensing,  noise  reduction,  rollover 
detection,  seat-belt  restraint  and  tensioning  and  more. 

MEMS'  debut  in  the  automotive  industry  led  to  varied  uses 
in  other  systems.  By  2011,  an  estimated  3  billion  MEMS  were 
used  to  heat  and  apply  ink  in  inkjet  printers.  Incorporated 
into  cameras,  these  tiny  machines  stabilize  optics  and  en¬ 
able  clear  photos,  even  when  held  by  unsteady  hands.  If  you 
have  played  Nintendo  Wii,  a  MEMS  in  the  hand-held  con¬ 
troller  senses  and  transmits  body  movements  to  the  video 
game— much  like  the  Lycra  suit  with  85  MEMS,  worn  by  ac¬ 
tors,  which  aided  computer-generated  imagery  in  the  movie 
"Iron  Man  2." 

In  the  biotech  industry,  MEMS  have  aided  micro-level  tasks. 
They  sense  blood  pressure,  monitor  glaucoma-related  eye 
pressure  and  pump  insulin  when  needed.  Additionally,  re¬ 
searchers  have  developed  MEMS  that  provide  a  high  degree 
of  control  over  stimuli  that  cells  receive  in  lab  environments. 

According  to  a  Yole  Development  report,  Status  of  the  MEMS 
Industry  2013,  the  MEMS  market  will  double  between  2012  and 
2018,  expanding  from  $11  billion  to  $22  billion.  That  expansion 
undoubtedly  will  include  increased  use  of  MEMS  in  munitions. 

The  Promise  for  Munitions 

In  movies,  the  term  "explosives  expert"  often  describes 
someone  with  competence  in  this  area.  To  some  degree,  that 
characterizes  the  Division  at  Indian  Head.  The  Division  does 
more,  however.  It  researches  and  develops  explosives,  propel¬ 
lants  and  pyrotechnics,  a  field  called  "energetics"  because  the 
materials  release  chemical  energy.  The  explosive  cartridges 
activating  car  air  bags  are  traceable  to  those  in  ejection  seats, 
developed  at  Indian  Head.  The  Division  also  develops  counters 
to  energetic  threats,  such  as  improvised  explosive  devices. 
Because  of  this  work,  the  Division  serves  as  a  Department  of 
Defense  (DoD)  Energetics  Center. 

Shortly  after  MEMS'  introduction  into  the  marketplace,  the 
Division  began  investigating  them  for  munitions.  It  did  what 
MIT  Professor  Judy  Hoyt  advocated:  "To  innovate  in  prod¬ 
ucts,  one  has  to  understand  the  processes  by  which  these 
products  are  made."  The  Division  set  up  a  clean  room  to 
characterize  and  fabricate  MEMS.  While  many  such  clean 
rooms  exist,  the  one  at  Indian  Head  EODTD  is  the  only  one 
certified  to  handle  and  integrate  explosives  with  MEMS,  in¬ 
cluding  firing  small  amounts  in  a  test  chamber. 

The  Division's  investigation  led  to  developing  MEMS  for  pos¬ 
sible  use  in  underwater,  sea  surface,  land  and  air-delivered 
munitions.  It  first  characterized  commercial  MEMS  for  the 
torpedo  U.S.  Navy  surface  ships  will  use  to  kill  enemy  torpe¬ 
does.  Within  this  6.75-inch-diameter,  anti-torpedo  torpedo, 
eight  MEMS  sensors  aid  navigation  by  measuring  accelera¬ 
tion,  angular  rate  and  flow,  thus  enabling  detonation  at  a  safe 
distance  from  the  launching  ship. 


At  just  $5  apiece,  the  first 
commercial  MEMS  reduced 
air-bag systems'jtosts  from 
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The  Division  also  developed  MEMS  fuzes.  Fuzes  have  been 
called  the  brains  of  munitions.  They  keep  a  munition  safe,  arm 
it  after  firing  and  then  detonate  it  at  an  opportune  time  relative 
to  a  target.  How  fuzes  do  this  varies.  Some  are  mechanical, 
detonating  on  impact.  Some  use  timers,  and  some  rely  on  a 
tiny  radar  to  detect  targets  and  then  detonate.  All  are  relatively 
big,  however.  For  example,  a  torpedo  fuze  is  about  the  size  of 
a  coffee  can.  Fuzes  also  take  up  a  fair  portion  of  a  munition's 
weight.  In  the  10-pound,  81mm  mortar  round,  the  fuze  weighs 
a  half-pound. 

The  MEMS  fuzes  produced  by  Indian  Head  EODTD  are  about 
fingernail  size,  with  mechanical  and  electronic  features  only  a 
tenth  of  a  hair's  width.  The  first  was  for  the  anti-torpedo  tor¬ 
pedo.  Although  it  uses  a  macro-size  fuze,  significantly  reduced 
in  size  by  the  Division,  its  experimental  MEMS  fuze  informed 
development  of  such  fuzes  for  more  widely  used  munitions. 

These  MEMS  fuzes  work  at  the  micro-level,  interfacing  with 
munitions  at  the  macro-level,  as  exemplified  by  one  developed 
for  mortars.  After  launch,  a  "G"  sensor  measures  accelera¬ 
tion  and,  when  a  given  number  of  Gs  is  reached,  removes  a 
safety  lock  in  the  MEMS,  the  first  step  toward  arming.  Another 
sensor  measures  the  round's  spin  rate  and,  at  a  set  number, 
signals  removal  of  a  second  lock,  arming  and  readying  the  mu¬ 
nition  for  a  fire  signal.  This  may  come  from  a  point-detonating 
switch,  a  delayed  detonation  setting,  or  another  sensor  detect¬ 
ing  a  target.  The  signal  initiates  a  tiny  explosive  in  MEMS'  first 
unlocked  section,  setting  off  another  explosive  in  the  second 
unlocked  section,  which  detonates  the  main  explosive. 

These  MEMS  fuzes  do  something  else:  harvest  energy.  They 
convert  launch  acceleration  into  electricity.  Thus,  MEMS  fuzes 
can  power  not  only  their  electronics  but  additional  MEMS  sen¬ 
sors.  They  can  be  power  sources  in  munitions,  which  previously 
lacked  them,  thus  allowing  incorporation  of  additional  functions. 
Energy-harvesting  MEMS  are  also  cheaper,  safer,  faster-activat¬ 
ing  and  longer-lasting  power  sources  than  batteries. 

MEMS  fuzes  also  free  up  space  in  existing  munitions.  This 
extra  space  can  be  used  for  improved  capabilities  and  addi¬ 
tional  functions. 
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MEMS  clean  room  at  Naval 
Surface  Warfare  Center 
Indian  Head  Explosive  Ord¬ 
nance  Disposal  Technology 
Division,  the  only  one  certi¬ 
fied  to  handle  and  integrate 
explosives  with  MEMS. 


Improved  Capabilities.  More  room  in  munitions  allows  ad¬ 
dition  of  more  propellant,  thus  increasing  range,  which  U.S. 
forces  in  Afghanistan  needed  for  mortars  and  shoulder- 
launched  munitions.  Another  option  would  be  to  add  more 
explosive,  thus  increasing  lethality.  For  example,  the  81mm 
mortar  round  carries  about  a  pound  of  explosive.  Replacing 
its  present,  half-pound  fuze  with  the  dime-size  MEMS  could 
increase  the  explosive  payload  by  almost  50  percent. 

Additional  Functions.  MEMS  fuzes  and  added  sensors  and 
technologies  may  provide  the  following  new  functions: 

■  More  fuzing  functions.  For  example,  today's  40mm  grenades 
detonate  only  on  impact.  Flowever,  MEMS  fuzes  could  delay 
detonation  until  after  impact.  More  sensors  could  be  added, 
such  as  those  with  radar  or  radio  frequency  (RF)  capabili¬ 
ties,  making  it  possible  to  detect  targets  and  detonate  in 
close  proximity. 

■  Faster  arming.  Warfighters  need  this  for  shoulder-launched 
munitions  in  close  engagements.  Their  time  between  firing 
and  impact  is  about  a  fifth  of  a  second.  That  means  fuzes 
must  arm  munitions  in  less  time,  which  MEMS  fuzes  can. 

■  Precision  guidance.  These  technologies  are  getting  smaller 
and  cheaper,  making  them  increasingly  available.  A  guid¬ 
ance  system  with  a  MEMS  fuze  soon  will  be  demonstrated 
in  81mm  and  120mm  mortars,  and  could  be  incorporated 
similarly  into  the  2.75-inch-diameter  Hydra  rockets 
launched  by  helicopters,  fixed-wing  aircraft  and  possibly 
Navy  surface  ships. 

MEMS  fuzes  also  can  endure  tough  environments.  Electron¬ 
ics  in  fuzes  for  medium-caliber  munitions,  such  as  30mm, 


experience  severe  shock  and  vibration  in  gun  barrels,  with 
acceleration  exceeding  60,000  Gs.  Also,  munitions  for  hard 
targets  must  have  fuzes  that  function  after  penetration, 
which  has  been  problematic.  MEMS  fuzes  have  sustained 
100,000  Gs  and  high-velocity  impacts  on  hard  targets,  mak¬ 
ing  them  suitable  for  missiles,  artillery,  naval  gunfire  rounds 
and  future  railgun  projectiles.  MEMS  sensors  already  are  in 
the  Extended  Range  Guided  Munition,  Hellfire  missile  and 
Small  Diameter  Bomb. 

Additionally,  MEMS  make  new  munitions  possible.  MEMS 
fuzes  and  sensors,  along  with  advances  in  explosives  and 
propellants,  can  lead  to  smaller,  lighter  and  more  powerful 
munitions,  which  is  the  trend  in  warfare.  Such  munitions  are 
sought  for  U.S.  ground  warfighters,  who  carried  more  than 
130  pounds  each  in  Afghanistan.  Attack  aircraft  have  gone 
from  carrying  2,000-pound  bombs  to  the  250-pound  Small 
Diameter  Bomb,  allowing  more  targets  to  be  hit  per  sortie, 
and  smaller  munitions  would  enable  even  more.  Furthermore, 
smaller  and  lighter  munitions  especially  are  needed  for  ever- 
smaller  unmanned  systems. 

Just  as  MEMS  do  incredibly  small  tasks  like  manipulating  cells 
in  medical  research,  they  can  do  the  incredibly  small  tasks  with 
respect  to  munitions  and  even  micro-vehicles.  Fuzes  could 
further  shrink  for  nano-energetics— nanometer-size  explosive 
and  propellant  particles— releasing  energy  faster  and  enabling 
very  powerful  and  very  tiny  munitions.  MEMS  also  may  be 
used  in  micro-thrusters  for  miniature  munitions  or  aerial  mi¬ 
cro-vehicles.  An  autonomous,  4-by-7-millimeter  micro-robot, 
using  nano-energetics  integrated  with  microchips,  already  has 
been  demonstrated. 
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MEMS  uses  even  go  beyond  detonating  munitions.  They  could 
help  reduce  traumatic  brain  injuries,  which  more  than  150,000 
U.S.  military  personnel  have  suffered  since  2000.  Most  of 
these  injuries  are  caused  by  blasts,  but  newly  developed 
MEMS  sensors,  which  can  be  worn  in  warfighters'  helmets, 
will  detect  the  blast  pressures  that  cause  brain  injuries.  Such 
quick  detection  will  alert  medical  personnel  in  time  to  arrest 
brain  cell  death  with  serums  and  other  treatments. 

Compelling  Reasons:  Reliability  and  Low  Cost 

"Munitions  system  reliability  must  be  addressed  soon,  oth¬ 
erwise  a  critical  aspect  of  our  warfighting  capability  will  be 
jeopardized  and  held  to  even  higher  levels  of  scrutiny,"  stated  a 
2005  Defense  Science  Board  report,  Munitions  System  Reliabil¬ 
ity.  It  also  stated,  "Fuzes  based  on  integrated  circuits,  Micro- 
Electro-Mechanical  Systems  and  integrated  fuzing,  targeting 
and  guidance  systems  can  provide  greater  reliability." 


which  is  currently  lacking  in  smaller  bomblet  and  submunition 
ordnance,"  stated  a  1995  DoD  assessment,  Microelectrome¬ 
chanical  Systems  Opportunities. 

MEMS  also  can  aid  reliability  by  helping  monitor  munitions' 
health.  Ordnance  is  designed  for  a  life  averaging  5  to  8  years, 
but  it  is  often  stockpiled  for  about  25  years.  Over  time,  ord¬ 
nance  reliability  can  be  affected  by  chemical  changes,  accel¬ 
erated  by  extreme  temperature  and  poor  handling.  Making 
matters  worse,  it  is  hard  to  tell  whether  it  is  still  good.  In  Op¬ 
eration  Iraqi  Freedom,  one  of  eight  Patriot  missile  containers 
was  dropped,  causing  concern  that  propellant  grains  or  guid¬ 
ance  components  may  have  been  damaged.  Yet,  the  dropped 
container  could  not  be  identified  as  it  had  no  visible  damage. 
Thus,  all  eight  were  sent  to  the  United  States  for  evaluation 
and  possible  repair— costing  $22  million. 

In  a  demonstration  in  Germany  and  Indiana  and  aboard  Navy 
ships,  MEMS  that  could  sense  such  drops  were  incorporated 
into  munitions  packages.  Their  data  and  other  data  were  col¬ 
lected  via  RF  tags  by  a  system  that  monitored  the  health  of  mu¬ 
nitions.  A  similar  idea  was  recommended  by  the  Defense  Sci¬ 
ence  Board's  Munitions  System  Reliability  report.  In  the  future, 
such  systems  conceivably  could  communicate  with  MEMS 
inside  munitions,  assessing  fuzes  and  other  health  aspects. 

In  a  time  of  austere  defense  budgets,  there  is  another  com¬ 
pelling  reason  for  munitions-related  MEMS:  low  costs.  After 
MEMS  commercial  debut,  the  DoD  report  Microelectrome¬ 
chanical  Systems  Opportunities  assessed  these  devices  for 
their  potential  cost  savings:  "A  $30,000  missile  typically 
contains  $1,000  worth  of  conventional  accelerometers  and 
gyroscopes.  An  equivalent  MEMS  device,  costing  $20,  can 
be  directly  substituted  in  this  platform.  This  represents  a  50x 
subsystem  cost  reduction."  Just  as  a  $10  MEMS  blood  pres¬ 
sure  monitor  replaced  a  $600  macro-size  one  in  the  medical 
community,  MEMS  can  provide  similar  savings  in  munitions 
across  defense  systems. 


On  1,400  identified  sites  alone,  "Estimated  clean-up  cost  of 
current  unexploded  ordnance  is  tens  of  billions  of  dollars," 
according  to  a  2003  Defense  Science  Board  report.  Of  great 
concern  is  unexploded  ordnance  from  cluster  munitions  in  the 
battlespace.  These  constitute  the  vast  majority  of  U.S.  indirect 
tactical  fires.  Some  manufacturers  claim  submunition  failure 
rates  of  2  percent  to  5  percent,  while  mine  clearance  special¬ 
ists  report  10  percent  to  30  percent  failure  rates,  according 
to  a  Congressional  Research  Service  report.  In  2008,  DoD 
policy  mandated  that  U.S.  forces  after  2018  will  employ  only 
cluster  munitions  "that  do  not  result  in  more  than  1  percent 
unexploded  ordnance  (UXO),"  further  stating,  "The  1  percent 
UXO  limit  will  not  be  waived." 

MEMS  fuzes  have  long  been  seen  as  capable  of  reducing  fail¬ 
ure  rates.  "MEMS  devices  offer  the  opportunity  for  5x  to  lOx 
greater  reliability,  performance,  and  service  life  through  im¬ 
proved  safe-arming/detonating  functions  and  inherent  quality, 


Looking  Ahead 

Industry  provides  a  model  for  MEMS  implementation.  "We 
are  seeing  a  massive  proliferation  of  MEMS  devices  across  a 
broad  range  of  applications:  from  mobile  handsets,  tablets  and 
pico  projectors,  to  health/medical  monitors,  automotive  safety 
systems,  the  smart  grid,  gaming,  and  robotics,"  said  Karen 
Lightman,  managing  director  of  the  MEMS  Industry  Group. 

That  proliferation  should  occur  in  defense.  MEMS  devices, 
particularly  fuzes,  can  help  add  functions  and  reliability  to 
munitions  and  other  defense  systems  at  low  costs.  In  doing 
so,  they  can  help  maintain  technological  advantage  for  U.S. 
warfighting  in  a  time  of  tight  budgets  and  growing  challenges. 
Especially  now,  defense  needs  the  MEMS  revolution  that's 
happening  throughout  commercial  industry.  & 


The  authors  can  be  contacted  at  daniel.jean@navy.mil,  kevin.r.cochran@ 
navy.mil,  and  daniel.pines@navy.mil. 
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Is  the  Key  "What"  or  "When"? 


Lt.  Gen.  Aloysius  G.  Casey,  USAF  (Ret.) 
Col.  Eugene  J.  DeNezza,  USAF  (Ret) 


Illustration,  Defense  Meteorological  Satellite  Program  orbiter.  Source:  U.S.  Air  Force  Space  and  Missile  Systems  Center. 


serious  discussion  is  under  way  within  the  defense  community  on  the  strategic  direction 
of  future  space  system  acquisitions.  Among  the  questions  being  addressed: 

Now  that  the  difficulties  with  our  major,  large,  aggregated  space  systems  seem  to  have  been  overcome, 
should  the  United  States  simply  continue  and/or  improve  these  systems  over  time?  Or  should  we  go 
quickly  toward  small-satellite,  platform-focused,  distributed-system  architectures? 


What  seems  to  be  missing  in  the  discussion  is  a  basic  tenet  of  good  acquisition  management:  Do  not  initiate  a 
major  acquisition  program,  either  a  revolutionary  architecture  change  or  an  evolutionary  one,  until  the  key  tech¬ 
nologies  are  mature. 


Casey  is  a  technical  and  management  consultant  specializing  in  space  systems ,  launch  vehicles ,  aircraft  unmanned  air  vehicles  and  as¬ 
sociate  weapons  systems.  He  capped  his  34-year  Air  Force  career  as  commander  of  the  Space  Division  of  the  Air  Force  Systems  Command , 
charged  with  managing  design ,  development ,  delivery  and  operation  of  the  Air  Force  space  systems  and  launch  vehicles.  DeNezza  is  a  senior 
associate  with  Burdeshaw  Associates,  Ltd.  For  more  than  17  years ,  he  served  in  top  management  at  TASC  (originally  The  Analytic  Sciences 
Corporation).  During  his  23-year  Air  Force  career ;  DeNezza  held  very  responsible  program  management  positions ,  including  deputy  program 
director ;  Strategic  Systems  Program;  commander ;  European  Office  of  Aerospace  Research  and  Development;  and  program  manager ;  Air 
Force  Advanced  Navigation  Programs. 
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What  We  Buy 

Certainly,  what  we  buy  is  very  important.  Successful  space  ac¬ 
quisitions  depend  on  many  factors.  For  a  military  space  system 
to  succeed,  mission  performance  requirements  must  be  met; 
costs  must  be  affordable;  the  system  must  be  available  when 
needed  (resilient);  and  it  must  be  adaptable  to  new  mission 
needs.  All  these  factors  influence  what  we  decide  to  buy. 

There  is  an  ongoing,  low-key  debate  within  the  space  com¬ 
munity  on  whether  to  move  to  proposed  small-satellite,  dis¬ 
tributed  architectures  or  to  continue  and/or  evolve  existing 
aggregated  systems.  In  a  recent  Strategic  Studies  Quarterly 
article,  "Disruptive  Challenges,  New  Opportunities,  and  New 
Strategies,"  Lt.  Gen.  Ellen  Pawlikowski,  Doug  Laverro,  and  Col. 
Tom  Cristler  argued  that  our  current  and  near-future  space 
systems  lack  the  needed  resiliency,  affordability  and  adapt¬ 
ability  demanded  by  new  international  realities.  These  new 
realities,  or  disruptive  challenges,  include: 

■  Widespread  and  growing  operational  dependence  on  U.S. 
space  systems. 

■  Growing  threats  (e.g.,  anti-satellite  weapons  (ASATS),  jam¬ 
ming,  cyber  attacks,  etc.)  to  these  systems. 

■  Poor  resiliency  of  U.S.  space  systems  (i.e.,  large,  expensive 
satellites;  few  if  any  spares;  small  constellations  and  easy 
targets). 

■  Fragility  of  U.S.  constellations.  A  loss  or  delay  of  a  single 
satellite  greatly  degrades  capabilities. 

■  Technological  stagnation  of  our  systems. 

■  The  shrinking  industrial  base,  especially  suppliers  in  the 
second  and  third  tier. 

In  formulating  a  response  to  these  disruptive  changes,  the  au¬ 
thors  argued,  "We  found  the  most  important  elements  were 
not  the  conditions  surrounding  what  we  build,  but  rather  the 
architectures  we  choose  to  build." 

They  further  concluded  that  many  challenges  are  a  direct  re¬ 
sult  of  building  aggregated,  highly  integrated,  long-lived  satel¬ 
lites.  The  solution  presented  concentrates  on  small-satellite, 
platform-focused,  distributed-system  architectures.  The  po¬ 
tential  advantages  they  advocated  for  this  revolutionary  archi¬ 
tectural  approach  fall  into  four  important  areas: 

■  Cost  and  Schedule  Improvement 

—  Lower  the  cost  of  individual  satellites. 

—  Use  less  costly  mission  assurance  and  smaller,  less-ex¬ 
pensive  launch  vehicles. 

—  Use  executable  baselines  (cost  and  schedule). 

—  Create  hosting  opportunities  at  reduced  cost. 

■  Industrial  Base  Strengthening 

—  Use  smaller  satellites  in  larger  constellations  that  call 
for  a  continuous,  multiyear  production  line,  thereby 
strengthening  the  industrial  base  and  lowering  cost. 

■  Improved  Resiliency 

—  Lower-cost  options  for  adding  on  orbit  spare  or  redundant 
systems— and  ground  reserves  for  reconstitution. 


—  Increase  constellation  size,  distribute  capability  and  re¬ 
duce  the  impact  of  losing  a  satellite  to  render  more  dif¬ 
ficult  an  attack  on  the  satellites. 

■  Ease  of  Technology  Insertion 

—  Use  less  complex  satellites  to  allow  for  easier,  new  tech¬ 
nology  insertion  and  capability  upgrades. 

This  concept,  however,  has  its  critics.  We  maintain  that  the 
transition  from  mission  to  architecture  focus  must  be  as¬ 
sessed  and  analyzed  carefully.  Once  requirements  are  de¬ 
fined,  architectural  alternatives  represent  only  one  metric. 
Other  needs  that  must  be  assessed  to  arrive  at  a  best  value 
program  include  acquisition  strategy,  sensor  performance, 
satellite  performance,  total  integrated  system  performance, 
launch  vehicle  requirements,  ground  station  architecture  and 
user  equipment. 

As  to  the  proposed  advantages  of  a  small-satellite,  platform- 
focused,  disaggregated  architecture,  other  considerations 
merit  discussion. 

Definitive  Analysis 

There  has  not  yet  been  a  rigorous  analytical  comparison  of 
using  a  proposed  small-satellite,  distributed  architecture  ver¬ 
sus  evolving  an  existing  aggregated  system  for  each  mission. 

Cost  and  Schedule 

Though  the  costs  of  the  existing  aggregated  systems  are  very 
high,  there  is  no  reason  to  believe  the  small-satellite,  distrib¬ 
uted  architectures  will  cost  less. 

The  need  for  aggregation  and  complexity  is  driven  by  mission 
performance  requirements.  Disaggregation  may  reduce  the 
cost  of  an  individual  satellite  but  not  necessarily  the  cost  of  the 
composite  architecture  needed  to  fulfill  the  mission. 

Many  smaller  satellites  in  an  architecture  have  unique  constel¬ 
lation  management  issues,  possible  constellation  intercom¬ 
munications  requirements,  transition  considerations,  ground 
infrastructure  complexities  and  user  costs  that  could  greatly 
increase  the  composite  architecture  life-cycle  costs. 

When  deciding  whether  to  evolve  from  the  existing  aggregated 
systems  (e.g.,  SBIRS,  AEHF,  etc.),  the  impressive  technology 
and  performance  advances  now  available  in  those  systems 
also  must  be  considered  and  treated  carefully. 

Hosted  payloads  may  have  a  role,  but  many  unique  challenges 
concerning  weight,  power,  space,  communications,  and  satel¬ 
lite  support  must  be  addressed. 

Space  System  Industrial  Base 

While  a  small-satellite  production  line  may  help,  other  cur¬ 
rent  factors  contribute  to  the  industrial  base  problem.  These 
include  the  reduced  opportunities  for  space-related  research 
and  development;  current  fiscal  downsizing;  budget  instability; 
the  inability  of  primes  (until  Efficient  Space  Procurement)  to 
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Right:  National  Polar-orbiting 
Operational  Environmental  Satellite 
System. 

Source:  National  Oceanic  and  Atmo¬ 
spheric  Administration. 

Below:  Advanced  Extremely  High 
Frequency  satellite  system. 

Source:  Lockheed  Martin. 


Technology  Insertion 

Disaggregation  of  a  satellite  system  doesn't  necessarily  mean 
it  is  easy  to  insert  new  technologies.  Complex  transition  man¬ 
agement  across  several  evolutionary  generations  and  con¬ 
figurations  of  space  payloads  still  could  be  costly  and  take 
considerable  time  to  implement  as  evidenced  by  the  recent 
Global  Positioning  System  (GPS)  program  experience. 


It  is  hoped  that  this  brief  treatment  of  the  very  complex  archi¬ 
tectural  issues  illustrates  there  are  no  easy,  obvious  architec¬ 
ture  solutions  and  that  architecture  is  only  one  of  many  space 
system  acquisition  issues.  One  of  the  most  important  is  when 
to  enter  the  Defense  Acquisition  System. 


block  buying  satellites;  and  the  prohibition  against  part  and 
subsystem  purchases  across  a  number  of  individual  programs. 

Resiliency 

There  seems  to  have  been  none  of  the  needed,  detailed  stud¬ 
ies  to  assess  the  relationship  between  constellation  size  and 
resilience  and  to  demonstrate  how  much  disaggregation  is 
enough.  Intuitively,  more  satellites  may  make  it  more  difficult 
to  attack  the  capability— but  not  necessarily  by  much. 

Other  space  protection  measures  should  factor  into  our  strat¬ 
egy.  Examples  include  international  agreements  and  treaties; 
satellite  self-defense;  decoys;  counterstrike  capability;  a  clearly 
stated  U.S.  policy  to  retaliate  for  any  such  attack;  space  control 
capabilities;  hardening;  and  augmentation  capabilities  from 
the  aerial  and  ground  levels.  Proliferation  of  satellites  may  be 
the  least  cost-efficient  path  to  resiliency. 


When  We  Buy 

There  are  some  very  important  lessons  in  our  space  history. 
When  we  interviewed  industry  and  government  executives 
and  asked  what  space  programs  they  considered  successful 
models  and  which  were  troubled,  the  answers  were  very  con¬ 
sistent.  Successful  programs  included: 

■  Discoverer/Corona  and  its  successors 

■  Transit 

■  Defense  Support  Program  (DSP) 

■  Defense  Satellite  Communications  System  (DSCS) 

■  Defense  Meteorological  Satellite  Program  (DMSP) 

Almost  as  many  were  considered  "troubled"  programs, 
including: 

■  Future  Imagery  Architecture  (FIA) 

■  Military  Strategic  and  Tactical  Relay  (MILSTAR) 
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■  National  Polar-orbiting  Operational  Environmental  Satel¬ 
lite  System  (NPOESS) 

■  Space  Based  Infrared  System  (SBIRS) 

■  Advanced  Extremely  High  Frequency  (AEHF)  satellites 

We  decided  to  look  at  these  programs  broadly  to  see  if  we 
could  discern  a  consistent  pattern  that  would  point  to  the  really 
important  differences  between  successful  and  troubled  pro¬ 
grams  in  the  context  of  our  present  situation. 

Lessons  from  Successful  Programs 
Discoverer/Corona 

The  Discoverer/Corona  (KH-1  through  KH-5/6)  programs 
produced  a  series  of  strategic  reconnaissance  satellites  used 
from  1963  through  1972.  The  program,  at  inception,  was  truly 
revolutionary.  This  is  demonstrated  by  the  number  of  firsts 
Discoverer/Corona  achieved.  Among  those  were  the  first 
polar  orbiting  satellite  (Discoverer  1);  the  first  3-axis  stabilized 
satellite  maneuverable  from  the  ground;  the  first  to  send  a 
re-entry  vehicle  back  to  Earth  (Discoverer  2);  the  first  suc¬ 
cessful  recovery  of  a  re-entry  vehicle  (Discoverer  13/KH-1); 


But  it  wasn't  easy  at  first.  Transit  went  through  five  experi¬ 
mental  satellites,  three  series  of  operational  prototypes,  and 
11  short-lived  "operational  satellites"  before  the  fully  success¬ 
ful  0-12  (Oscar  12).  The  design  through  this  period  evolved. 
Navigation  accuracy  went  from  120  meters  (1964)  down  to 
3  meters  (1980). 

Defense  Support  Program  (DSP) 

The  DSP  is  one  of  the  most  successful  U.S.  programs.  It  is  a 
survivable,  reliable  satellite  constellation  designed  to  detect 
missile  and/or  spacecraft  launches  and  nuclear  explosions. 
The  DSP  has  undergone  five  technology  upgrades  that  have 
taken  its  weight  from  2,100  to  5,250  pounds;  its  power  from 
400  watts  to  over  1,250  watts;  its  detectors  from  2,000  to 
more  than  6,000;  and  its  design  life  from  1.25  years  to  more 
than  5  years.  Since  its  first  launch  in  1970,  DSP  has  provided 
40  years  of  uninterrupted  space-based  early  warning. 

Defense  Satellite  Communications  System  (DSCS) 

The  DSCS  program  began  in  1967  with  the  launch  of  three 
Initial  DSCS  I  satellites.  The  DSCS  I  program  launched  27  initial 


■  v 

-  The  mere  fact  that  a  satellite 
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system  Is  disaggregated 
doesn't  necessarily  mean  it 
is  easy  to  insert  new  technologies. 


and  the  first  successful  recovery  of  image  intelligence  from 
space  (Discoverer  14/KH-1). 

These  "firsts"  came  at  a  price— 11  of  the  first  12  launches  were 
failures.  But  as  the  Corona  satellites'  technology  evolved,  per¬ 
formance  improved,  and,  by  the  end  of  the  program  in  1972, 
the  program  had  launched  144  satellites  with  102  successful 
recoveries  of  usable  photographs. 

Transit  (also  known  as  NAVSAT) 

Transit  was  the  first  operational  satellite  navigation  system. 
It  started  development  in  1959,  but  the  first  Transit  failed  to 
reach  orbit.  Transit  IB  was  launched  successfully  in  1960.  This 
was  followed  by  42  more  launches,  culminating  in  Transit 
Oscar-31  in  1988. 

Following  some  difficult  initial  problems,  the  Transit  proved 
very  reliable.  The  first  production  run  (Transit  Oscar-12 
through  Oscar-32)  of  this  indestructible  satellite  was  able  to 
keep  the  constellation  operational  for  more  than  32  years,  and 
the  constellation  still  is  transmitting  as  the  Navy  Ionospheric 
Monitoring  System. 


DSCS  satellites  with  one  failure.  The  DSCS  I  satellites  weighed 
100  pounds  and  contained  a  single  X-band  transponder. 

The  DSCS  II  program  was  approved  in  1968,  with  the  first 
launch  in  1971.  The  DSCS  II  satellites  were  a  significant  up¬ 
grade  of  DSCS  I.  This  1,150-  to  1,350-pound  satellite  empha¬ 
sized  hardening,  anti-jam  protection  and  increased  channel 
capacity.  The  communication  payload  included  two  20-watt 
X-band  channels.  Fifteen  DSCS  II  satellites  were  launched, 
with  two  failures. 

The  DSCS  III,  first  launched  in  1982,  remains  the  workhorse 
of  the  U.S.  military's  super  high-frequency  communications 
system.  It  offers  significantly  greater  capacity,  longer  life 
and  better-protected  communications  than  its  predeces¬ 
sors.  It  is  a  2,580-pound  satellite  with  six  channels  of  X- 
band  communications.  Fourteen  DSCS  III  satellites  have 
been  launched  successfully. 

Defense  Meteorological  Satellite  Program  (DMSP) 

Another  of  the  most  successful  U.S.  space  programs,  the 
DMSP  was  initiated  in  1961  at  the  National  Reconnaissance 
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Office  (NRO).  Now  in  its  fifth  decade  of  service,  the  DMSP 
still  provides  valuable  weather  data  to  the  military,  civil  and 
scientific  communities.  The  DMSP-5D3  is  the  latest  (11th) 
version  of  DMSP  satellites.  The  DMSP  has  evolved  from  a 
90-pound,  spin-stabilized  satellite  with  shutter-style  TV 
cameras  to  the  current  2,640-pound  satellite  with  seven 
sophisticated  instruments.  Fifty-one  DMSP  satellites  have 
been  launched  with  nine  failures  during  its  50-year  lifetime. 

Lessons  from  Troubled1  Programs 
Future  Imagery  Architecture 

A  book  could  probably  be  written  about  this  program— called 
by  the  New  York  Times  "perhaps  the  most  spectacular  and  ex¬ 
pensive  failure  in  the  50-year  history  of  American  spy  satellite 
programs."  In  summary,  NRO  decided  to  develop  optical  and 
radar  imagery  satellites  that  were  smaller,  lighter  and  less  ex¬ 
pensive  than  the  current  satellites.  Conceptual  architectural 
studies  began  in  1996,  but  it  wasn't  until  1999  that  Boeing 
was  awarded  the  optical  and  radar-imaging  satellite  contracts. 
Boeing  had  underbid  Lockheed  Martin  by  a  billion  dollars.  It 
was  a  very  surprising  selection  since  Lockheed  Martin  had 
supplied  all  the  then-current  imaging  spacecraft— optical  and 
radar— and  Boeing  had  never  built  the  kind  of  satellites  the 
government  was  seeking.  By  2005,  an  estimated  $10  billion 
had  been  spent,  twice  the  original  estimate  of  $5  billion.  Most 
analysts  believe  FI  A  was  destined  to  fail  because  the  technol¬ 
ogy  needed  to  meet  requirements  wasn't  mature,  there  wasn't 
enough  funding,  the  schedule  was  unrealistic  and  the  selection 
criteria  and  source  selection  process  for  the  space  element  of 
FIA  were  flawed. 

MILSTAR 

The  MILSTAR  program  officially  started  in  1981  to  develop 
a  secure,  jam-resistant,  worldwide  communication  satellite 
system.  The  first  launch  was  scheduled  for  1987.  Schedule 
slips  and  cost  overruns  started  in  1984  and  continued.  In  1991, 
DoD  restructured  the  program  by  reducing  constellation  size 
from  eight  to  six,  reducing  ground  stations  from  25  to  nine,  cut¬ 
ting  total  terminal  quantity  from  1,721  to  1,467  and  eliminating 
survivability  features. 

Six  satellites  were  launched  between  1994  and  2003,  with 
one  failure.  These  satellites  provide  jam-proof,  UHF  and 
high-data-rate  communications.  The  cost  of  reaching  the 
redirected  capability  has  been  estimated  at  $22  billion  (up 
from  an  estimated  $9  billion  to  $10  billion),  with  each  satel¬ 
lite  costing  about  $800  million.  The  schedule  slipped  more 
than  4  years. 

The  Government  Accountability  Office  identified  the  following 
MILSTAR  problems: 

■  Technology  was  insufficiently  mature  (concurrency). 

■  Software  needs  were  poorly  understood. 

■  Requirements  were  defined  inadequately. 

■  There  were  myriad  requirements  and  engineering 
changes. 


National  Polar-orbiting  Operational  Environmental 
Satellite  System  (NPOESS) 

NPOESS,  a  revolutionary,  very  complex,  next-generation 
weather  satellite  system  was  designed  to  monitor  the 
Earth's  weather,  atmosphere,  oceans,  land  and  near-space 
environment.  The  NPOESS  program  was  managed  jointly  by 
the  U.S.  Air  Force,  the  National  Oceanic  and  Atmospheric 
Administration  (NOAA)  and  NASA.  The  program  was  can¬ 
celed  Feb.  1,  2010,  due  to  cost  overruns,  schedule  slips  and 
technology  difficulties. 

The  NPOESS  program  was  an  effort  to  integrate  the  capa¬ 
bilities  of  the  NOAA  Polar-orbiting  Environmental  Satel¬ 
lite,  the  DoD  DMSP,  and  NASA's  continuous  climate  data 
collection  satellite  into  one  satellite.  Some  of  the  NPOESS 
problems  centered  on  technically  maturing  its  large  suite 
of  very  sophisticated  sensors. 

This,  coupled  with  many  interagency  management  problems, 
killed  the  program. 

Space-Based  Infrared  System  (SBIRS) 

Like  the  NPOESS,  the  SBIRS  was  conceived  in  an  era  when 
the  prevailing  wisdom  called  for  combining  missions  on 
a  single  satellite  to  reduce  the  number  of  satellites  and 
launches,  saving  development  and  operational  costs.  The 
SBIRS  satellites  were  built  to  satisfy  four  missions— missile 
warning,  missile  defense,  technical  intelligence  and  battle- 
space  characterization. 

The  program  encountered  significant  technical  problems  (both 
hardware  and  software),  unclear  requirements,  unexpected 
software  complexity  and  unstable  funding.  As  a  result,  pro¬ 
gram  costs  ballooned  and  the  schedule  slipped  dramatically. 
The  program  now  faces  parts  and  subsystem  obsolescence 
challenges.  If  the  government  decides  to  purchase  GEO  6  and 
7,  the  focal  plane  array  substrate  will  have  to  be  replaced,  as 
the  only  company  that  made  the  substrate  material  has  gone 
out  of  business.  This  problem  largely  came  about  because  of 
the  multiyear  schedule  slippages  but  also  because  some  of 
the  parts  are  unique  to  SBIRS  and  have  only  a  single  supplier 
or  no  source. 

The  recently  launched  SBIRS  GEO  and  HEO  satellites  are  per¬ 
forming  very  well  and  provide  significantly  improved  perfor¬ 
mance  and  utility  to  their  users. 

Advanced  Extremely  High  Frequency  (A EHF) 
Satellite  System 

The  AEHF  is  a  planned  six-satellite  constellation  to  be  used  to 
relay  service  communications  for  U.S.,  British,  Canadian  and 
Dutch  military  forces  worldwide.  Two  of  the  six  have  been 
launched  and  are  undergoing  tests.  AEHF  will  provide  10  to 
12  times  the  bandwidth  and  6  times  the  data  rates  and  it  will 
support  twice  as  many  tactical  networks  as  the  MILSTAR  II 
satellites.  This  means  the  AEHF  will  deliver  10  to  12  times  the 
data  throughput  of  MILSTAR  so  that  "for  every  one  link  of  the 


13 


Defense  AT&L:  January-February  2014 


Illustration  of  a  disaggregated  satellite  system.  Source:  Defense  Advanced  Research  Projects  Agency. 


old  MILSTAR,  the  Air  Force  now  has  12  links  operating  at  4  (to 
6)  times  the  speed." 

The  AEHF  had  its  share  of  technical  problems,  including  inter¬ 
face  control  redesigns,  delayed  delivery  of  signal-encryption 
products,  disqualified  parts,  and  unplanned  component  test¬ 
ing.  But  the  program  suffered  as  much,  if  not  more,  from  as 
many  as  six  changes  in  the  requirements  and  number  of  sat¬ 
ellites,  from  budget  fluctuations  and  from  constant  program 
replanning  and  rebaselining.  Two  successful  AEHF  launches 
have  occurred,  and  testing  is  under  way.  The  satellites  have 
displayed  very  impressive  performance  in  these  early  tests. 

Summary 

Lessons  we  might  justifiably  draw  from  this  brief  and  broad 
recap  of  past  satellite  programs  include: 

(1)  Virtually  all  the  programs,  successful  or  troubled,  that  were 
revolutionary  in  technology  and/or  design  had  significant  ini¬ 
tial  cost,  schedule  and  technical  problems  (see  Discoverer/ 
Corona,  Transit,  NPOESS,  SBIRS  and  AEHF). 

(2)  Once  the  technology  matures  (e.g.,  Discoverer/Corona, 
Transit,  MILSTAR,  SBIRS,  and  AEHF),  revolutionary  systems 
can  further  evolve  and  improve  with  significantly  fewer  prob¬ 
lems  as  long  as  the  inserted  technology  is  mature  and  has  a 
continuing  industrial  base.  For  example,  to  go  from  the  Dis¬ 
cover/Corona  re-entry  capsules  to  the  first  real-time  imaging 
satellite,  the  risk  reduction/maturation  process  took  about 


The  most  successful  pro¬ 
grams  (DSP,  DSCS  and 
DMSP)  really  were  evo¬ 
lutionary  programs.  That 
is,  the  initial  versions  were  challenging  technically  but  didn't 
require  major  technological  leaps.  These  programs  had  some 
initial  problems  but  these  were  much  less  significant  than 
those  seen  in  the  more  revolutionary  programs.  Significant 
upgrades  were  made,  with  few  major  problems,  in  subsequent 
versions— when  the  technology  was  mature. 

It  is  clear  that  evolution  of  a  mature  technology  (be  it  from 
the  beginning  of  the  full-scale  development  effort  or  after  the 
painful  maturing  of  a  revolutionary  development  effort)  is  the 
best  approach  for  successful  space  system  acquisition. 

One  key  to  successful  space  system  development  is  to  initiate 
acquisition  of  the  operational  space  system  after  the  research- 
and-development  effort  has  matured  the  technology  to  be  uti¬ 
lized.  Maturity  can  be  defined  as  follows:  The  technology  has 
been  developed,  tested  on  the  ground  and  in  orbit;  production 
sources  have  been  identified  and  costs  verified;  and  perfor¬ 
mance  ranges  (i.e.,  marginal  performance  vs.  cost)  have  been 
established.  Our  bottom  line  is  that  the  United  States  should 
evolve  its  present  systems  carefully.  Evolutionary  changes 
should  be  made  as  the  technology  matures,  and  revolutionary 
architecture  changes  should  be  deferred  until  small-satellite, 
distributed-system  technology  has  been  thoroughly  analyzed, 
developed  and  tested;  costs  have  been  verified;  performance 
ranges  established  and  production  sources  identified. 


The  authors  can  be  reached  at  agcasey1@verizon.net  and  ejdenezza@ 
aol.com. 


5  years  and  more  than  $1 
billion  (in  today's  dollars). 

(3)  Technical  maturity 
must  be  matched  by  a 
significant  production 
base  for  the  parts  and 
subsystems  used  on  the 
satellites.  As  SBIRS  is 
demonstrating,  if  there  is 
no  production  base  and 
no  commercial  adjunct, 
the  Service  would  not  only 
bear  the  cost  of  develop¬ 
ing  the  system  but  as  sole 
customer  would  have  to 
support  the  manufactur¬ 
ing  base  at  enormous  cost 
over  the  many  years  of  a 
typical  system  life  cycle 
or  develop  new  technol¬ 
ogy  to  replace  the  parts 
or  subsystems. 
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DoD's  2014  Research  a: 


For  more  than  50  years,  the  Department  of  Defense  (DoD)  has  relied  on  Re¬ 
search  and  Engineering  (R&E)  to  provide  the  nation  with  the  technology-based 
operational  capability  superiority  that  protects  U.S.  forces  and  helps  to  ensure 
national  security.  DoD's  scientists  and  engineers  work  daily  with  industry  and 
academia  to  conceive,  develop  and  mature  concepts  into  capabilities  that 
provide  an  operational  advantage  to  our  warfighters. 


The  Office  of  the  Assistant  Secretary  of  Defense  for  Research  and  Engineering  (OASD[R&E])  is  responsible  for 
the  department's  current  and  future  technical  and  engineering  capabilities  and  for  helping  to  define  the  technical 
boundaries  and  expand  the  realm  of  the  possible  early  in  the  department's  acquisition  process.  The  office  includes 
the  roles  of  chief  technology  officer,  chief  engineer,  and  chief  of  developmental  testing,  and  is  responsible  for  lead¬ 
ing  the  rapid  transition  of  new  technologies  to  the  warfighter. 


Shaffer,  Acting  Assistant  Secretary  of  Defense  for  Research  and  Engineering  (ASD[R&EJ),  is  responsible  for  formulating,  planning,  and 
reviewing  the  Department  of  Defense  (DoD)  Research,  Development,  Test,  and  Evaluation  (RDT&E)  programs,  plans,  strategy,  priorities 
and  execution  of  the  DoD  RDT&E  budget. 
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The  security  of  our  nation  and  the  capabilities  of  our  war¬ 
fighters  depend  on  a  robust  partnership  of  research  and  de¬ 
velopment  (R&D)  and  acquisition  efforts.  In  the  past  decade, 
examples  of  fielded  capabilities  delivered  to  the  warfighter 
in  part  due  to  the  efforts  of  the  OASD(R&E)  include:  the 
first  Mine  Resistant  Ambush  Protected  vehicle  system,  which 
provides  dramatically  greater  protection  for  its  passengers; 
aerostat  systems  that  deliver  24/7  surveillance  around  our 
forward  operating  bases;  a  laser  radar  system  that  maps 
topography  at  a  very  high  resolution  and  identifies  objects 
under  canopy;  sensors  and  platforms  that  enable  use  of 
full-motion  video  in  support  of  operational  and  intelligence 
needs;  portable  hybrid  photovoltaic/battery  systems  supply¬ 
ing  needed  power  at  much  lower  weight;  and  wound  repair 
and  tissue  regeneration  capability. 

Simply  put,  the  job  of  the  OASD(R&E)  is  to  address  the  cur¬ 
rent  and  future  needs  of  the  DoD  and  its  warfighters.  As  our 
warfighters  move  off  the  frontline  in  Iraq  and  Afghanistan, 
and  the  department  rebalances  to  the  Asia-Pacific  region, 
continued  investment  is  needed  in  innovative  and  adapt¬ 
able  capabilities  to  meet  the  challenges  of  emerging  global 
threats. 

In  his  foreword  to  the  Defense  Department's  January  2012 
strategy  report,  "Sustaining  Global  Leadership:  Priorities 
for  21st  Century  Defense,"  President  Obama  wrote:  "As  we 
end  today's  wars  and  reshape  our  Armed  Forces,  we  will 
ensure  that  our  military  is  agile,  flexible,  and  ready  for  the 


■  Mitigate  new  and  emerging  adversary  threats  that  could 
degrade  U.S.  (and  allied)  capabilities. 

■  Affordably  enable  new  or  extended  capabilities  in  existing 
military  systems. 

■  Develop  technology  surprise  through  science  and  engi¬ 
neering  applications  to  military  problems. 

These  objectives  complement  the  seven  science  and  technol¬ 
ogy  (S&T)  priorities  approved  in  2011  by  then-Secretary  of 
Defense  Robert  Gates.  The  focus  to  meet  these  objectives  is 
provided  by  the  collective  effort  of  the  Services  and  defense 
agencies  in  cyber,  electronic  warfare,  countering  weapons  of 
mass  destruction,  engineered  resilient  systems,  autonomy, 
data  to  decisions,  and  human  systems. 

Mitigate  Emerging  Threats 

The  Defense  Department  must  be  prepared  to  meet  its  cur¬ 
rent  and  future  missions  against  a  backdrop  of  increasing 
types  and  complexities  of  foreign  systems  and  capabilities. 
The  following  is  an  overview  of  threat  areas  and  how  the  R&E 
community  is  addressing  these  challenges  to  maintain  and 
advance  our  military's  capabilities. 

Cyber:  We  are  integrating  our  efforts  in  cyber  defense  across 
the  department  and  have  prioritized  our  R&D  investments 
in  several  focus  areas:  the  hardening  and  expansion  of  cyber 
test  ranges;  developing  metrics  for  cyber  events;  resilience 
under  attack;  improving  network  agility  to  reduce  the  avail¬ 
able  target:  and  the  modeling  and  simulation  of  cyber  nets. 


we  will  continue  to  invest  in  the  capabilities  critical  to  future 
including  intelligencer/4u/JeiillnVer  and  reconnaissance; 


success, 

counterterrorism;  cotin 


of  mass 'destruction 


operating  in  anti-access  environments;  and  prevailing  in  all  domains 


including  cyber. 


President  Barack  Obama,  January  2012 


full  range  of  contingencies.  In  particular,  we  will  continue  to 
invest  in  the  capabilities  critical  to  future  success,  including 
intelligence,  surveillance,  and  reconnaissance;  counterter¬ 
rorism;  countering  weapons  of  mass  destruction;  operating 
in  anti-access  environments;  and  prevailing  in  all  domains, 
including  cyber." 

Research  &  Engineering  Objectives 

The  guidance  is  clear:  The  president  and  the  secretary  of  de¬ 
fense  rely  on  the  R&E  community  to  make  key  contributions 
to  the  defense  of  our  nation.  This  guidance  is  summarized  in 
three  strategic  objectives: 


Advanced  Electronic  Warfare  Systems:  The  proliferation 
and  global  availability  of  advanced  digital  electronics  cou¬ 
pled  with  high-speed  processing  and  advanced  algorithms 
are  enabling  adversaries  to  develop  effective  countermea¬ 
sures  to  our  sensor  and  communications  capabilities.  We 
are  working  to  ensure  that  the  technology  capabilities  we 
see  around  the  world  will  not  undercut  the  effectiveness  of 
some  of  our  most  promising  and  sophisticated  sensors  and 
weapons  systems.  The  department's  emphasis  in  countering 
these  threats  is  focused  on  maturing  and  testing  the  next 
generation  of  electronic  and  photonic  components  as  well 
as  expanding  the  operating  capabilities  of  existing  and  new 
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electronic  systems,  to  maintain  and  increase  our  advantage  effectiveness  of  fielded  systems  and  provide  life-cycle  costs 
in  the  electromagnetic  spectrum.  for  decision  making. 


Space:  The  U.S.  military  relies  on  dependable  global  position¬ 
ing  systems,  communications  and  intelligence,  surveillance 
and  reconnaissance.  A  focus  of  our  efforts  is  protection  of 
these  critical  capabilities  against  emerging  threats,  such  as 


The  tools  and  procedures  of  ERS  will  produce  more  com¬ 
prehensive  and  robust  requirements  suitable  for  many  more 
alternative  mission  scenarios  very  early  in  the  design  process 
or  pre-Milestone  A.  The  reuse  of  data  and  models,  distrib- 
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their  impact  on  all  design  and  production  factors. 


electronic  jammers  and  dazzlers,  and  kinetic  kill  vehicles, 
which  present  a  challenge  to  future  operations.  We  must  be 
able  to  operate  freely  even  in  austere  or  compromised  en¬ 
vironments.  The  chip-scale  atomic  clock  provides  precision 
navigation  and  timing  so  that  every  dismount  can  know  its 
position.  We  also  are  developing  alternative  communications 
networks  that  do  not  rely  on  satellites. 

Affordably  Enable  New  or  Extended  Capabilities 

The  second  strategic  objective  emphasizes  maximizing  our 
investment  dollars  by  improving  the  design  and  transition  of 
technologies  to  acquisition  programs  in  a  more  affordable 
and  timely  manner.  Our  focus  is  on  both  new  and  existing 
systems  and  their  life-cycle  upgrades,  interoperability  be¬ 
tween  existing  platforms,  and  design  and  prototyping  of  new 
systems.  Our  Engineering  Resilient  Systems  program,  the 
enhanced  use  of  prototypes,  and  our  "Shift  Left"  initiative 
all  help  guide  our  efforts. 

Engineered  Resilient  Systems  (ERS):  Our  focus  in  ERS  is  to 

develop  engineering  concepts,  science,  and  design  tools  to 
protect  against  malicious  compromise  of  weapon  systems 
and  to  greatly  enhance  the  manufacturability  of  trusted  and 
assured  defense  systems  across  the  acquisition  life  cycle. 
Through  the  ERS  initiative,  the  department  is  developing 
an  integrated  suite  of  computational  modeling  and  simula¬ 
tion  capabilities  and  systems  engineering  tools,  complete 
with  an  open-reference  architecture,  directly  aligned  to 
acquisition  and  operational  business  processes.  The  R&D 
investment  in  ERS  focuses  on  infrastructure,  information, 
design  support,  highly  robust  tradespace  analytics,  decision 
support  tools  and  knowledge  environments  to  increase  the 
speed  and  efficiency  of  system  development,  improve  the 


uted  databases  that  are  searched  jointly,  virtual  collaboration 
environments  and  open  architectures  that  encourage  part¬ 
nering  will  lead  to  better-informed  acquisition  decisions.  The 
engineering  design  process  is  streamlined,  and  the  manu¬ 
facturability  of  a  proposed  design  is  explicitly  investigated 
from  both  engineering  and  cost  perspectives  before  design 
commitment.  Finally,  we  are  developing  robust  tools  to  stress 
systems  against  new  mission  contexts,  tactics,  techniques 
and  procedures  or  emerging  requirements,  to  permit  precise 
measurement  and  understanding  of  their  impact  on  all  design 
and  production  factors. 

Capability  Prototyping:  Operational  prototyping  is  a  flex¬ 
ible  and  robust  tool  we  intend  to  use  to  a  greater  extent  in 
evaluating  new  systems  and  technologies,  stimulating  de¬ 
sign  teams  and  exploring  the  realm  of  the  possible  without 
needing  an  early  commitment  to  procurement.  Capability 
protoyping  allows  us  to  develop  and  demonstrate  concepts 
and  technologies  at  varying  degrees  of  maturity  as  a  hedge 
against  the  technical  uncertainty  of  an  unanticipated  threat, 
to  enhance  the  interoperability  or  extend  the  utility  of  existing 
systems  and  to  enable  the  introduction  of  more  capability 
and  affordability.  It  is  our  intent  to  employ  capability  pro¬ 
totyping  more  for  subsystems  and  force  multipliers,  but  we 
expect  to  do  some  platform  prototyping  as  well. 

Shift-Left  Initiative:  Affordability  also  can  be  addressed  by 
engaging  with  an  acquisition  program  earlier  in  the  life  cycle 
to  ensure  a  better  understanding  of  program  technical  risks 
and  opportunities  before  major  milestone  decisions.  Early 
systems  engineering  and  Developmental  Test  and  Evaluation 
(DT&E)  will  better  identify  and  correct  deficiencies,  thereby 
avoiding  the  high  cost  of  late-cycle  repairs. 
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The  Shift-Left  initiative  is  about  improving  the  effectiveness 
of  DT&E,  so  programs  can  find  and  fix  problems  before  enter¬ 
ing  production.  Effective  DT&E  also  requires  using  a  robust 
DT&E  framework  to  ensure  a  program  conducts  the  right 
testing  at  the  right  time  to  gather  the  required  information 
before  milestone  decisions.  There  are  three  essential  ele¬ 
ments  of  the  initiative: 

■  Mission  Context:  Bringing  mission  context  into  DT&E 
takes  the  new  system  out  of  the  lab  to  see  how  it  will  be 
used.  It  does  not,  however,  mean  program  managers  have 
to  conduct  developmental  testing  with  large-scale  deploy¬ 
ment  of  troops  to  the  field. 

■  Interoperability:  Throughout  the  last  decade  of  combat 
operations,  we  did  not  find  interoperability  issues  early 
enough  to  correct  them  before  system  deployment. 

■  Cybersecurity:  Ten  years  of  data  from  Director  of  Op¬ 
erational  Testing  and  Evaluation  field  tests  assessing 
cybersecurity  indicate  that  fielded  systems  must  be 
protected  in  the  cyber  domain.  Vulnerabilities  should  be 
discovered  in  testing  and  corrected  before  deployment. 

Technology  Surprise:  Develop  New  Capabilities 

While  we  will  need  to  react,  we  must  focus  on  leading  change. 
Thus,  our  third  strategic  objective,  and  a  critical  component 
in  the  department's  ability  to  develop  new  capabilities,  is  to 
invest  in  a  wide  range  of  potentially  game-changing  basic  and 
applied  research.  Autonomy  is  one  of  three  emphasis  areas 
that  include  data  to  decisions  and  human  systems  priority 
areas,  where  our  integrated  efforts  across  the  Services  will 
provide  new  capabilities. 

Autonomy  Research  Pilot  Initiative:  Autonomous  technol¬ 
ogy  is  just  one  of  the  new  capability  areas  that  will  enable 
technology  surprise.  Our  activities  focus  on  the  ability  to 
enable  DoD  warfighting  systems  to  be  more  adaptable  and 
more  self-governing,  to  function  with  greater  independence 
from  human  interaction  and  with  reduced  response  times  in 
stressed  environments. 

To  complement  our  contracted  research  with  industry  and 
academia,  we  established  the  Autonomy  Research  Pilot  Initia¬ 
tive,  a  new  process  to  evaluate  internal  innovative  ideas.  This 
effort  is  a  coordinated  program  among  DoD  laboratories,  en¬ 
couraging  cross-Service  collaboration  by  bench-level  scientists 
and  engineers  on  autonomy-related  applied  research  topics. 

Guided  by  feedback  from  operational  experience  and  evolv¬ 
ing  mission  requirements,  seven  proposals  were  selected 
out  of  54  submissions  to  be  a  part  of  this  multiyear  funded 
activity.  Advancement  of  technologies  will  result  in  autono¬ 
mous  systems  that  provide  more  capability  to  warfighters, 
lessen  the  cognitive  load  on  operator/supervisors  and  lower 
overall  operational  cost.  In  addition,  the  investment  in  this 
cross-Service  collaboration  will  strengthen  mission  effective¬ 
ness  while  maintaining  fiscal  responsibility  and  optimizing 
interoperability  across  all  domains. 


The  Future:  R&E’s  Coordinated  Approach 

A  key  strength  of  our  R&E  enterprise  is  its  substantial  empha¬ 
sis  on  coordinated  planning.  Especially  in  these  challenging 
budgetary  times,  it  is  important  to  strengthen  the  depart¬ 
ment's  efforts  to  ensure  that  we  receive  the  most  value  from 
our  investment.  As  part  of  this  coordinated  approach,  we 
have  taken  steps  to  provide  new  tools  to  communicate  our 
efforts  both  internally  and  with  industry  and  academia. 

Horizon  Scanning:  To  conduct  effective  technology  horizon 
scanning,  we  are  developing  a  low-cost  effort  to  apply  ad¬ 
vanced  analytics,  leveraging  a  range  of  algorithms  and  data 
streams,  to  isolate  and  identify  emerging  science  and  tech¬ 
nology  areas. 

DoDTechipedia:  The  ability  to  bring  together  performers 
and  operational  and  acquisition  communities  is  a  challenge. 
DoDTechipedia  (www.dtic.mil)  hosts  an  online  electronic 
encyclopedia  and  provides  a  platform  where  organizations 
can  share  information  on  challenges  and  needs. 

Defense  Innovation  Marketplace:  A  critical  source  of  in¬ 
novation  technology  comes  from  outside  the  department. 
The  key  to  this  innovation  source  is  communication,  thus 
we  deployed  the  Defense  Innovation  Marketplace  (www. 
DefenselnnovationMarketplace.mil)  to  enable  better  con¬ 
nections  between  industry  and  the  DoD.  The  Marketplace 
provides  S&T/R&D  technology  needs,  investment  priorities 
and  roadmaps  to  help  industry  members  better  support  the 
warfighter  through  their  independent  research  and  develop¬ 
ment  (IR&D)  projects.  In  addition,  the  site  contains  a  secure 
portal  for  industry  to  share  its  R&D  projects  and  for  DoD 
personnel  to  learn  about  these  industry  IR&D  projects  to 
better  inform  current  and  future  program  planning. 

The  DoD  will  always  rely  on  R&D  to  provide  the  nation  with 
technical  superiority.  Our  investment  over  the  last  50  years 
has  enabled  many  breakthroughs  including  stealth,  GPS, 
unmanned  technology,  Web  protocols,  advanced  robotics 
and  data  mining. 

As  we  move  into  this  new  year,  with  the  current  budget  pres¬ 
sures  and  a  new  security  landscape,  we  must  continue  to 
balance  our  investments  across  the  department  as  well  as 
with  our  acquisition  partners.  Moreover,  the  last  two  major 
budget  contractions  (post-Vietnam  and  the  ending  of  the 
Cold  War)  showed  that  strategic  efforts  by  the  department's 
leadership  protected  R&D  and  allowed  us  to  make  important 
advances  in  maintaining  our  technological  edge.  The  value 
of  our  research  and  development  investments,  and  focus  on 
new  tools  that  improve  affordability  and  communication, 
strengthen  our  overall  competencies  so  we  can  continue  to 
provide  capabilities  and  value  to  the  warfighter.  & 
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A  Program  Master  Schedule 
Can  Improve  Results 

Patrick  K.  Barker 


“There  cannot  be  a 
crisis  today;  my  schedule 
is  already  full/' 

—Henry  Kissinger 

Scheduling  from  a  Government  Perspective 

Schedules  are  important.  As  government  acquisition  officials,  we  want  things  delivered 
at  the  time  we  agreed  they  would  be  delivered  and  for  what  we  agreed  to  pay.  We 
want  to  know  whether  something  is  going  to  be  late  or  over  cost.  Contractor-delivered 
schedules  can  help  the  government  program  manager  (PM)  answer  some  of  these 
important  questions.  However,  a  contractor  schedule  rightly  focuses  only  on  contract 
scope  and  not  on  the  entire  program  picture. 

A  successful  government  PM  must  maintain  and  share  situational  awareness  (SA)  across  the  entire  program 
and  its  environment.  This  requires  visibility  beyond  contract  scope,  from  budget  drills  to  congressional  inquiries 
to  warfighter  requirement  changes  and  everything  in  between.  Government  acquisition  programs  are  complex 
systems  highly  sensitive  to  emerging  conditions  and  dependencies  among  multiple  elements.  It  is  an  excep¬ 
tionally  challenging,  but  not  impossible,  task  for  a  government  PM  to  maintain  SA.  In  other  words,  it  is  awfully 
hard  to  keep  track  of  everything,  let  alone  predict  what  might  happen  at  any  point  in  time. 

A  well-built  and  believable  dynamic  model  that  is  of  practical  use  provides  a  powerful  tool  to  support  PM  deci¬ 
sion  making.  To  enable  this,  the  government  program  management  office  (PMO)  can  employ  its  own  program- 
level  schedule.  To  distinguish  this  from  the  contractor's  schedule,  let's  call  this  a  "PMO  Integrated  Master 
Schedule  (IMS)''  or  "PMO  IMS.''  A  PMO  IMS  helps  the  government  PM  to  maintain  SA  and  make  informed 
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decisions.  While  this  article  won't  be  the  "final  word"  on  a 
PMO  IMS,  it  will  highlight  desirable  PMO  IMS  characteristics 
and  important  PMO  considerations. 

Desired  Characteristics 

A  dynamic  model  reflective  of  best  scheduling 

practices 

We  build  models  to  represent  reality  in  ways  we  can  under¬ 
stand  and  use  for  making  decisions.  Typical  acquisition  mod¬ 
els  are  used  to  depict  aircraft  flight,  radar  propagation,  heat 
signatures,  reliability,  cost  relationships  and  so  on.  A  schedule 
models  a  program  by  depicting  the  duration  of  and  relationship 
among  the  program-specific  tasks  that  constitute  the  required 
work.  Since  actual  performance  will  differ  from  parts  of  the 
initial  model,  this  schedule  must  also  be  dynamic  or  change¬ 
able  in  order  to  characterize  the  program  accurately  over  time. 

Thus  a  PMO  IMS  must  be  built  in  such  a  way  that  it  reasonably 
represents  the  program  and  can  keep  pace  with  its  evolution. 
Use  of  proper  schedule  construction  mechanics— like  those 
described  in  references  such  as  the  Notional  Defense  Indus¬ 
trial  Association  Planning  and  Scheduling  Excellence  Guide ,  the 
Defense  Contract  Management  Agency's  "Fourteen-Point 
Assessment,"  and  the  draft  Government  Accountability  Office 
Schedule  Assessment  Guide— lays  the  groundwork  for  effective 
dynamic  modeling  of  programs.  These  are  frameworks  forged 
by  subject  matter  experts  who  successfully  applied  schedul¬ 
ing  to  project  management.  Inattention  to  best  practices  like 
these  puts  the  schedule  on  the  fast  track  to  irrelevance,  at  best 
out  of  date  before  the  ink  dries.  Poorly  constructed  schedules 
become  little  more  than  "to  do"  lists  or  expensive  calendars 
incapable  of  providing  SA. 

Well-built  schedules  can  depict  more  than  program  status. 
They  are  also  predictive.  Schedules  help  the  government 
PM  anticipate  changes  and  consider  "what-if"  scenarios  on 
demand,  because  dynamic  modeling  allows  individual  task 
dependencies,  current  status  and  individual  forecasts  to  be 
overlaid  and  projected  in  a  program  context.  Commercial 
scheduling  software  is  uniquely  capable  of  doing  this  in  re¬ 
peatable  and  reliable  fashion.  However,  scheduling  software 


doesn't  always  produce  the  "pretty"  pictures  preferred  for 
briefings,  so  PMOs  often  rely  on  static  pictures  and  Gantt 
charts  available  on  presentation  software  such  as  Microsoft 
PowerPoint.  That's  fine— and  often  necessary— for  clear 
communication.  However,  over-reliance  on  eye-pleasing 
graphics  risks  projecting  a  false  impression  of  SA  over  a  dy¬ 
namic  process. 

Believability 

Robust  dynamic  modeling  lays  the  groundwork  for  an  effective 
PMO  IMS,  but  there  is  more  work  to  do,  and  it  takes  more  than 
the  schedulerto  make  it  happen.  Program  team  members  and 
stakeholders  most  familiar  with  the  work  and  program-unique 
environment  are  best  suited  to  judge  how  accurately  an  IMS 
depicts  the  program.  These  same  individuals  also  need  to  as¬ 
sess  external  information  affecting  the  PMO  IMS,  such  as  that 
coming  from  the  contractor. 

A  scheduler  should  sit  down  and  talk  with  other  team  mem¬ 
bers;  the  scheduler's  initiative  and  interpersonal  skills  go  a 
long  way  toward  ensuring  the  PMO  IMS  remains  accurate  and 
relevant  over  time.  However,  team  interaction  is  a  two-way 
street.  The  other  PMO  members  must  help  create,  scrub  and 
continually  update  the  schedule  in  order  to  trust  the  informa¬ 
tion  it  provides.  A  team-built  and  team-operated  PMO  IMS 
replaces  a  program  picture  characterized  by  multiple  individ¬ 
ual  snapshots  with  a  single  picture  composited  from  various 
contributors.  Team  ownership  paves  the  way  for  "buy-in"  of 
schedule  information. 

Practicality 

The  schedule  ought  to  provide  actionable  answers  to  ques¬ 
tions  like  these: 

■  What  work  must  be  done,  and  when? 

■  Who  is  doing  the  work,  and  when? 

■  What  is  happening  right  now? 

■  Where  are  we  going? 

■  What  risks/opportunities  do  we  face  ahead? 

■  When  will  we  be  done,  and  how  do  we  know? 

Answering  the  above  questions  requires  inputs  from  and 
integration  of  a  variety  of  sources  in  order  to  ensure  that 
schedule  information  provides  robust  SA.  These  include  re¬ 
quirements  documents,  statements  of  work,  product  work 
breakdown  structures,  organizational  breakdown  structures, 
risk  register  inputs,  integrated  master  plans  and  technical 
performance  measures,  to  name  a  few.  Schedulers  with  a 
strong  project  management  background  are  critical  because 
they  grasp  the  individual  contributions  of  these  sources,  rec¬ 
ognize  when  information  from  a  given  source  is  missing  or  of 
questionable  value,  and  take  action  to  correct  the  situation 
without  prompting. 

The  PMO  IMS  does  not  need  to  be  huge,  but  it  should  "pull" 
information  that  enhances  the  government  PM's  SA  at 
any  given  time.  A  contractor  might  produce  a  15,000-line 
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schedule  in  order  to  cover  its  contract  scope  and  associated 
tasks,  but  the  associated  government  PMO  IMS  might  only  be 
200  to  300  lines.  A  PMO  IMS  might  expand  in  some  sections 
to  provide  detailed  insight  into  high  risks  but  remain  more  gen¬ 
eral  for  low  risk  areas.  It  will  also  expand  and  contract  in  size 
over  time  as  the  program  evolves.  External  events  or  inputs 
that  might  influence  the  program  are  always  included,  along 
with  key  risk-handling  efforts  and  decision  points.  Getting  a 
usable  PMO  IMS  off  the  ground  takes  thought  and  hard  work, 
but  it  should  not  take  weeks  of  effort  at  inordinate  expense. 


PMO  Considerations 

Integrating  schedules  is  complex  but  rewarding. 

A  lead  PMO  scheduler  typically  faces  integrating  schedule 
information  from  multiple  sources,  including  contractors 
and  stakeholders.  If  a  PMO  integrates  related  but  different 
schedules  produced  by  different  people,  then  it  creates  a 
scheduling  system.  Integration— especially  automated  in¬ 
tegration— is  easier  said  than  done.  Therefore,  a  PM  should 
consider  the  following: 


■  A  common  program  and  scheduling  language 

■  Shared  desktop  procedures  (how  an  individual  creates  and 
maintains  a  schedule) 

■  Compatible  data  used  in  the  various  "fields"  within  schedul¬ 
ing  software 

■  Identical  scheduling  software  (Project,  Primavera,  Open 
Plan,  etc.) 

■  Shared  ideas  of  what  constitutes  healthy  schedules  (e.g., 
practical,  believable,  dynamic  models) 


Unsuccessful  scheduling  systems  drive  schedulers  to  spend 
much  of  their  time  diagnosing  and  fixing  schedules,  which 
means  little  attention  is  spent  analyzing  and  evaluating  pro¬ 
gram  execution.  This,  in  turn,  jeopardizes  the  PMO  IMS  ability 
to  support  the  PM. 


Schedule  integration  is  a  systemic  challenge,  and  it  therefore 
requires  a  systemic  response.  Paying  attention  to  the  individual 
considerations  shown  above  is  one  thing.  Enforcing  all  of  them 
across  a  diverse  set  of  organizations  and  schedulers  is  an¬ 
other  matter  entirely.  Server-based  scheduling  tools  can  help 
to  a  degree,  but  they  do  not  compensate  for  poor  scheduling 
discipline,  lack  of  believability  and/or  poor  communication. 
As  the  saying  goes,  garbage  in  equals  garbage  out.  On  the 
other  hand,  successful  multi-schedule  integration  is  a  power¬ 
ful  "force  multiplier"  for  program  SA.  It  frees  valuable  time  for 
analysis,  synthesis  and  evaluation  of  schedule  information.  As 
a  result,  effective  PMO  IMS  information  can  be  made  available 
in  time  for  a  PM  to  make  proactive  decisions. 


Knowledge  of  a  commercial  scheduling  tool  does 
not  solely  define  the  PMO  IMS  scheduler. 

True,  a  scheduler  should  be  familiar  with  the  nuances  of  the 
scheduling  software  used  by  the  program  and  have  a  firm 
grasp  of  scheduling-discipline  fundamentals.  It  takes  time  to 
develop  a  skilled,  professional  scheduler  based  on  published 


professional  guidelines:  There  simply  is  no  way  around  that. 
Expectations  for  professional  schedulers  are  best  defined 
within  the  context  of  industry  certifications  such  as  the  Ameri¬ 
can  Association  of  Cost  Engineers  Planning  and  Scheduling 
Professional  and  the  Project  Management  Institute  Scheduling 
Professional.  However,  schedulers  are  not  simply  "tool  jock¬ 
eys."  Schedulers  also  must  have  a  firm  command  of  project 
management  theory  and  practice,  because  the  PMO  IMS  must 
be  both  believable  and  of  practical  use  to  the  government  PM. 
Leadership  and  interpersonal  and  communication  skills  in  the 
lead  scheduler  also  are  critically  important. 


A  good  PMO  IMS  scheduler  has  his/her  "finger  on  the  pulse" 
of  the  program  at  all  times,  and  doing  so  requires  an  individual 
with  a  balanced  set  of  skills.  It  is  important  for  a  scheduler 
to  be  adept  with  scheduling  software  and  discipline,  but  not 
entirely  at  the  expense  of  the  other  demands  of  the  job.  At 
times,  it  is  far  more  important  to  have  the  PMO  team  sitting 
around  a  table  engaged  in  thoughtful  and  facilitated  discussion 
on  schedule  risk  than  having  the  scheduler  spending  those 
hours  figuring  out  how  to  resource  load  the  PMO  IMS.  The 
PM  should  not  think  of  the  scheduling  position  as  something 
that  can  be  filled  only  by  an  experienced,  certified  scheduling 
professional.  There  is  more  to  scheduling  than  running  soft¬ 
ware.  The  PM  should  consider  filling  the  scheduling  position 
with  a  talented  individual  as  soon  as  possible  and  "growing" 
that  individual  overtime  in  scheduling,  project  management 
and/or  leadership  as  required. 


The  effectiveness  of  the  PMO  IMS  ultimately 
hinges  on  the  government  PM’s  leadership. 

The  government  PM  must  lead  the  PMO  team  to  use  and 
maintain  a  PMO  IMS  effectively.  This  requires  the  following: 
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■  The  PM  must  ensure  that  the  lead  scheduling  position  is 
staffed  by  a  high-quality  individual  who  either  already  has 
the  requisite  skill  or  in  whom  the  PM  is  willing  to  invest  time, 
resources,  and  mentoring  to  develop  the  necessary  skills. 

■  The  PM  should  know  what  key  questions  need  asking  to 
ensure  that  best  scheduling  practice  is  employed.  This  re¬ 
quires  the  PM  to  be  conversant  (not  a  subject  matter  expert) 
in  salient  aspects  of  schedule  construction  and  discipline. 

■  The  PM  must  lead  the  team  and  ensure  the  schedule  is  built 
and  maintained  in  a  joint,  multifunctional  fashion.  This  might 
involve  guiding  team  members  outside  their  comfort  zones. 

■  The  PM  must  proactively  shape  the  schedule  as  a  decision 
support  tool,  rather  than  be  a  passive  observer  and/or  re¬ 
cipient  of  scheduling  information.  The  PM  should  work  with 
the  scheduler  to  "pull"  the  information  needed  to  make  in¬ 
formed  decisions. 

■  The  PM  must  also  empower  the  scheduler  to  create  an  ef¬ 
fective  scheduling  system.  The  scheduler  does  not  have  a 
single  "swim  lane."  The  scheduler  swims  in  every  lane.  Build¬ 
ing  a  scheduling  system  often  requires  crossing  boundaries, 
and  a  lead  scheduler  needs  the  trust  and  authority  of  the  PM 
to  make  that  happen. 

No  matter  how  believable  and  practical  the  PMO  IMS  might 
be,  it  has  little  value  if  the  government  program  team  does 
not  use  it.  The  PMO  IMS  must  be  "owned"  by  the  govern¬ 
ment  PM  because  the  program  team  will  typically  take  their 
cue  from  the  PM.  For  example,  consider  a  PMO  IMS  schedule 
risk  analysis  (SRA).  An  SRA  should  never  be  performed  alone 
by  the  scheduler.  An  effective  SRA  requires  the  scheduler, 
team  leads,  subject  matter  experts  and  risk  owners  to  work 
in  concert;  the  PM  should  set  the  tone  for  that  effort.  The  PMO 


MDAP/MAIS  Program  Manager  Changes 


With  the  assistance  of  the  Office  of  the  Secretary  of  De¬ 
fense,  Defense  AT&L  magazine  publishes  the  names  of  in¬ 
coming  and  outgoing  program  managers  for  major  defense 
acquisition  programs  (MDAPs)  and  major  automated  infor¬ 
mation  system  (MAIS)  programs.  This  announcement  lists 
all  such  changes  of  leadership,  for  both  civilian  and  military 
program  managers  for  the  months  of  September  and  Octo¬ 
ber  2013.  (Some  program  managers  listed  took  the  position 
earlier  than  September  but  were  missed  by  the  Service  in 
their  previous  input.) 

Navy/Marine  Corps 

Capt.  Michael  N.  Abreu  assumed  the  position  of  program 
manager  for  the  Naval  Enterprise  Networks  Program  (PMW 
205)  on  Oct.  7. 

Karen  Davis  relieved  Capt.  Patricia  Gill  as  program  man¬ 
ager  of  Advanced  Technology  (PEO  [IWS])  on  June  10. 


IMS  needs  to  be  a  prominent  fixture  within  the  PM's  decision 
process  and  part  of  the  language  used  by  the  PMO  teams  to 
convey  status,  predictions,  strategies,  risks  and  opportunities. 

So  What? 

A  government  PM  needs  a  mechanism  to  collect  and  sift 
through  important  tactical  information  in  order  to  remain  fo¬ 
cused  on  the  big  picture.  To  borrow  from  aviation  parlance, 
you  cannot  maintain  effective  SA  if  your  head  remains  buried 
inside  the  cockpit.  If  the  PMO  IMS  provides  robust  and  ac¬ 
tionable  answers  to  key  performance  questions— How  are  we 
doing?  When  are  we  done?  What  lies  ahead?— the  PM  does 
not  have  to  spend  an  inordinate  amount  of  time  trying  to  inves¬ 
tigate  and  answer  those  questions.  This  frees  time  for  the  PM 
to  think  and  act  strategically,  "pulling"  PMO  IMS  information 
as  needed  to  support  SA  and  decision  making. 

The  PMO  IMS  is  a  communication  tool  that  helps  the  PM 
make  informed  decisions  and  enables  the  government  PMO 
to  maintain  a  proactive  stance.  While  this  article  offered  a 
set  of  considerations,  rarely  will  all  be  achieved  at  once  in  a 
PMO.  It  takes  effort  to  develop,  and  keep,  talented  schedul¬ 
ers  who  create  and  maintain  robust  schedules,  but  that's 
not  a  reason  to  delay  investment  of  time  and  energy  into  a 
PMO  IMS.  It  must  be  kept  in  mind  that  building  and  sustain¬ 
ing  a  PMO  IMS  is  not  an  end  in  itself.  Ultimately,  a  sched¬ 
ule  must  help  the  PM  make  decisions  about  the  program. 
Thus,  a  simple  but  useful  schedule  today  is  better  than  the 
detailed  and  perfect  schedule  next  month.  Time  is  money. 
Make  the  PMO  IMS  a  critical  contributor  to  decision  making 
in  your  program  office.  & 

The  author  can  be  contacted  at  patrick.barker@dau.mil. 


John  M.  Gamer  relieved  Col.  John  K.  Buckles  as  program 
manager  of  Advanced  Amphibious  Assault  (PEO  [LS])  on 
July  10. 

Air  Force 

Col.  Christopher  Coombs  relieved  Col.  Shaun  Morris  as  the 

AC  AT  ID  systems  program  manager  for  the  KC-46  Tanker 
Modernization  Program  on  Oct.  1. 

Col.  William  Leister  relieved  Col.  Christopher  Coombs  as 

the  ACAT  1C  systems  program  manager  for  the  MQ-1  & 
MQ-9  Unmanned  Aircraft  Systems  Programs  on  Oct.  1. 

Lt.  Col.  Kevin  Sellers  relieved  Lt.  Col.  Brian  McDonald  as 

the  ACAT  ID  program  manager  for  the  Three-Dimensional 
Expeditionary  Long-Range  Radar  (3DELRR)  Program  on 
Aug.  2. 

Col.  Gregg  Kline  relieved  Col.  Margaret  Larezos  as  the 

ACAT  IA  systems  program  manager  for  the  Air  and  Space 
Operations  Center  Weapon  System  (AOC  WS)  Increment 
10.2  Program  on  July  31. 
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for  Improved  Software  Reliability 


Timothy  Pohiand  ■  David  Bemreuther 


Software  reliability  poses  a  significant  challenge  for  the  Army  as  software  is  increasingly 
important  in  a  broad  range  of  applications.  The  safety,  welfare  and  effectiveness  of  our 
Soldiers  directly  depend  on  the  ability  of  software  to  perform  as  intended  and  operate 
reliably  in  adverse  and  austere  conditions.  The  ability  of  the  Defense  Department  (DoD)  to 
provide  a  high  level  of  Soldier  services  while  minimizing  overhead  and  other  sustainment 
costs  is  tied  directly  to  the  reliability  of  large  and  complex  software  systems. 

Software  is  a  key  enabler  in  compiling  intelligence,  conducting  analysis,  and  performing  command-and-control 
functions.  For  example,  the  Army  must  support  numerous  command,  control,  communications,  computers,  intel¬ 
ligence,  surveillance  and  reconnaissance  (C4ISR)  systems  at  a  cost  in  excess  of  $200  million  annually.  Furthermore, 
embedded  software  has  become  an  essential  feature  of  virtually  all  hardware  systems.  This  necessitates  assess¬ 
ing  system  reliability  through  a  holistic  accounting  of  hardware,  software,  operator  and  their  interdependencies. 


Pohiand  is  a  computer  engineer  and  a  member  of  the  U.S.  Army  Materiel  Systems  Analysis  Agency  ( AMSAA )  Reliability  Branch  with  5 
years  of  reliability  analysis ,  software  development ,  and  process  improvement  experience.  Bernreuther  is  an  operations  research  analyst 
at  AMSAA  Reliability  Branch  at  Aberdeen  Proving  Ground ,  Md.r  and  has  30  years  of  software  development ,  technical  analysis  and  process 
improvement  experience. 
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Figure  1.  Sample  Element  Description 
(Low  Risk) 


Design  for  Reliability 
Developmental  Testing 


■  All  modules  of  software  are  covered  by  unit  testing 
and  all  are  included  in  integration  testing. 

■  All  external  systems  are  included  (or  surrogated)  in 
integration  testing. 

■  Multiple  sets  of  test  data  are  available  to  support 
both  unit  testing  and  integration  testing  during  de¬ 
velopment.  The  test  data  adequately  represents  the 
scale  of  operations  that  the  software  will  encounter 
when  used  operationally. 


Issues  with  the  performance  and  reliability  of  software 
throughout  the  DoD  led  to  development  of  the  Capability 
Maturity  Model  (now  Capability  Maturity  Model  Integration 
or  CMMI)  in  the  late  1980s.  Similarly  industry  standards  such 
as  the  IEEE/ISO  12207  also  have  emerged  to  address  sound 
software  practices.  Additionally  a  plethora  of  tools  and  tech¬ 
niques,  from  requirements  management  tools  to  dynamic  code 
analyzers,  are  available  to  support  the  engineering  of  reliable 
software.  Yet,  as  mature  as  software  reliability  practices  and 
standards  have  become,  reliability  remains  a  significant  issue 
for  government  and  industry.  A  recent  sample  of  combined 
hardware-software  systems  tested  by  the  Army  illustrates  the 
significance  of  the  issue.  It  found  software  failures  pervasive 
throughout  the  sampled  systems,  constituting  52  percent  of 
the  overall  failures  (worst  case:  82  percent). 

Benefits  of  a  Scorecard  Approach 

Software  development  often  is  complex  and  expensive,  while 
available  resources  and  time  are  generally  limited.  This  exa¬ 
cerbates  the  challenges  of  ensuring  appropriate  and  effective 
practices  are  followed.  To  facilitate  development  of  more  reli¬ 
able  software,  the  Army  Materiel  Systems  Analysis  Agency 
(AMSAA)  has  developed  a  software  reliability  scorecard.  This 
instrument  extends  and  complements  an  existing  scorecard 
on  general  reliability  practices.  It  also  complements  organi¬ 
zation-centric  approaches,  such 
as  CMMI,  by  assessing  the  level 
of  risk  associated  with  reliability- 
specific  practices  within  an  indi¬ 
vidual  software  project. 

The  scorecard  is  a  structured  and 
transparent  instrument  for  as¬ 
sessing  the  health  of  an  individ¬ 
ual  software  development  effort 
and  is  invaluable  in  isolating  weak 
areas  for  further  analysis  and 
work.  It  enables  scarce  resources 


to  be  prioritized  and,  subsequently,  more  reliable  software  to 
be  developed.  The  research,  discussion  and  reflection  under¬ 
taken  while  applying  the  instrument  can  be  as  valuable  as  the 
resultant  score(s).  It  provides  a  common  structure  for  multiple 
disciplines  to  see  the  interrelationship  and  importance  of  vari¬ 
ous  reliability  issues  and  practices  outside  their  own  domain. 

The  scorecard  focuses  on  areas  that  warrant  additional  re¬ 
search  and  analysis.  It  highlights  areas  of  weakness  and, 
through  the  evaluator's  recommendations,  gives  insight  on 
how  to  address  those  weaknesses.  However,  it  is  not  pre¬ 
scriptive  as  to  specific  actions  that  should  be  taken.  AMSAA 
recommends  that  the  areas  of  concern,  once  discovered,  be 
investigated  further  to  identify  the  best  software  and  reliability 
practices  and  tools  to  be  applied  at  those  areas. 

How  It  Works 

The  software  reliability  scorecard  adheres  to  conventions 
similar  to  the  existing  General  Reliability  Scorecard,  which  is 
used  by  numerous  DoD  organizations.  It  is  self-contained  in  an 
Excel  spreadsheet  with  57  specific  elements  to  be  evaluated 
and  rated.  All  the  elements  are  grouped  into  seven  catego¬ 
ries,  along  with  definitions  for  each  rating  level,  and  laid  out 
in  a  single  input  sheet.  Each  element  is  rated  red,  yellow  or 
green  to  represent  high,  medium  or  low  risk,  respectively.  An 
example  of  a  rating  definition  of  Low  Risk  for  Developmental 
Testing  appears  in  Figure  1.  Additionally,  cells  are  provided 
for  the  rater  to  enter  a  rationale  and  recommendations  for 
each  element.  These  offer  important  insights  for  the  feedback 
process,  helping  turn  the  scorecard  ratings  into  focused  and 
effective  actions. 

The  scorecard  processes  the  individual  ratings  and  derives 
a  total  score  assessing  how  much  the  program  is  at  risk.  The 
ratings  are  adjusted  by  weighting  factors  assigned  to  each  in¬ 
dividual  element.  The  overall  risk  assessment  is  normalized 
to  a  value  between  1  and  100,  where  1  is  low  risk  and  100  is 
high  risk.  A  top-level  quantitative  assessment  is  illustrated  in 
Figure  2. 

The  scorecard  also  generates  summaries  of  each  of  the  seven 
categories  to  help  focus  on  strengths  and  weaknesses.  These 
pictorially  represent  the  number  of  high-,  medium-  and  low- 
risk  elements  in  each  category,  as  shown  in  Figure  3. 


Figure  2.  Sample  Overall  Rating 


Assessed 

Risk 


Normalized  Risk  Score  (1  to  100  scale  where  100  is  the  highest  risk)  =  54 


Overall  Risk  Assessment 


Low  Risk  Medium  Risk  High  Risk 
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Figure  3.  Sample  Category-Level 
Summary 

Design  for  Reliability 
Number  of  Ratings  by  Risk  Level 


Normalized  Risk  Score  (1  to  100  scale  where  100  is  the  highest  risk)  =  40 


It  is  highly  recommended  that  a  multidisciplinary  team  apply 
the  scorecard  to  a  project.  Initially,  each  member  should  assess 
the  project  from  his  or  her  perspective  and  knowledge.  The 
team  then  should  compare  and  discuss  its  members'  ratings 
of  each  element  to  achieve  consensus.  These  deliberations 
usually  are  as  insightful  and  valuable  as  the  rating  scores.  The 
team  should  use  the  Rationale  and  Recommendation  fields  to 
help  record  the  key  points  of  its  deliberations  and  draft  recom¬ 
mended  actions. 


Categories  and  Elements 

The  software  reliability  scorecard  addresses  seven  key  cat¬ 
egories  of  reliability  practices  most  applicable  to  software 
development:  program  management,  requirements  man¬ 
agement,  design  capabilities,  system  design,  design  for  re¬ 
liability,  (customer)  test  and  acceptance,  and  fielding  and 
sustainment.  Each  category  then  covers  a  number  of  specific 
elements  focusing  on  key  reliability  practices.  A  detailed  ex¬ 
amination  of  every  element  within  the  scorecard  is  beyond 
the  scope  of  this  article.  However,  a  discussion  of  each  of  the 
categories  and  some  of  their  key  features  should  provide  an 
understanding  of  the  breadth  and  depth  of  the  examination 
of  good  reliability  practices. 

Program  Management 

Development  of  reliable  software  requires  that  limited  re¬ 
sources  and  time  be  well  managed;  that  the  customer  and 
team  work  collaboratively;  and  that  reliability-enhancing  ac¬ 
tivities  receive  the  necessary  resources,  visibility  and  priority. 
The  Program  Management  category  addresses  these  needs 
by  looking  at  12  elements: 

■  Developer's  Experience 

■  Process  Maturity 

■  Program  Planning 

■  Currency  of  Plans 

■  Progress  Reviews 

■  Readiness 

■  Reliability  Engineering 

■  Reliability  Growth  Management 

■  Verification  and  Validation  (V&V) 

■  Supporting  Disciplines 

■  Risk  Management 

■  Commercial  Off-the-Shelf  (COTS)  Management 


The  instrument  is  designed  to  be  of  value  at  any  stage  of 
software  development.  Although  many  of  the  elements  lend 
themselves  to  specific  stages  of  the  software  life  cycle,  the 
results  are  most  insightful  when  all  elements  have  been 
assessed  regardless  of  the  project's  current  status.  If  the 
instrument  is  being  applied  later  in  a  project's  life  cycle,  as¬ 
sessment  of  the  earlier  activities  provides  valuable  insight 
about  the  character  of  the  existing  design  and  code.  Con¬ 
versely,  activities  that  are  usually  a  priority  later  in  a  project 
(preparations  for  Fielding  and  Sustainment,  for  example)  are 
"leading  indicators"  of  the  quality  and  thoroughness  of  the 
reliability  tests  being  conducted.  These  later  elements  are 
not  weighted  as  highly  as  earlier  activities  but  still  reduce 
risk.  A  project  that  proactively  starts  them  early  is  reducing 
the  overall  risk  of  the  project. 

There  may  be  cases  where  an  element  simply  does  not  apply 
to  a  given  project.  For  these  circumstances,  there  is  a  "not 
evaluated"  option  for  reviewers;  the  scorecard  will  drop  these 
elements  from  its  calculations  and  provide  a  normalized  risk 
score  from  the  remaining  elements.  In  the  summary  chart  for 
each  category,  a  count  of  the  "NE"  elements  is  provided. 


CMMI,  or  similar  organization  maturity  levels,  is  considered 
part  of  the  developer's  experience.  As  organizational  maturity 
does  not  guarantee  success,  this  element  also  considers  the 
developer's  experience  with  applications  of  the  domain,  size 
and  complexity  under  consideration.  Supporting  Disciplines 
include  human  factors,  technical  writing,  and  others— es¬ 
sential  to  the  software,  but  not  necessarily  involved  with  a 
project's  core  design  and  coding  activities. 

Requirements  Management 

Any  successful  development  project  relies  on  complete  and 
clearly  understood  requirements.  These  should  include  well- 
considered  and  -defined  reliability  goal(s).  A  good  system  for 
recording  requirements  and  linking  them  to  the  design  capa¬ 
bilities  that  will  provide  them  is  also  essential.  The  elements 
in  Requirements  Management  are: 

■  System  Requirements 

■  Currency 

■  Reliability  Goals 

■  Requirements  Allocation 

■  Quality  of  Requirements 
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Ideally,  the  requirements 
of  a  software  project  are 
defined  completely  and  well 
understood  when  design 
begins.  They  remain  static 
thereafter.  Realistically, 
this  rarely  happens  because 
requirements  evolve  while 
the  project  progresses. 


■  Use  Cases 

■  Interoperability 

■  Dependency 

■  Other  Characteristics 

Ideally,  the  requirements  of  a  software  project  are  defined 
completely  and  well  understood  when  design  begins.  They 
remain  static  thereafter.  Realistically,  this  rarely  happens  be¬ 
cause  requirements  evolve  while  the  project  progresses.  Ac¬ 
cordingly,  Requirements  Management  also  assesses  how  well 
the  project  realizes  requirements  have  changed  and  adapts  its 
plans  and  activities.  A  mature  developer  will  seek  to  discover 
and  accommodate  the  changes.  This  not  only  reduces  risk 
but  minimizes  costs  and  time,  as  resources  are  more  wisely 
allocated  to  accommodate  the  newest  requirements. 

Use  Cases  is  a  notable  element,  because  these  cases  pro¬ 
vide  a  powerful  method  to  enhance  understanding  between 
the  customer  and  developer  and  among  the  various  involved 
disciplines.  They  also  facilitate  requirements  definition,  de¬ 
velopment,  testing  and  documentation  such  as  users  guides. 
Their  use  greatly  aids  the  development  of  reliable  and  effec¬ 
tive  software. 

Design  Capabilities 

By  assessing  the  developer's  Design  Capabilities,  the  score- 
card  seeks  to  ensure  that  sufficient  capabilities  to  design  reli¬ 
able  software  are  in  place  and  implemented  effectively.  The 
seven  elements  of  Design  Capabilities  are: 

■  Development  Process 

■  Process  Implementation 


■  Documentation  and  Repository 

■  Configuration  Management 

■  Collaboration  Capabilities 

■  Development  Samples 

■  Analysis  of  Alternatives 

The  Development  Samples  element  is  particularly  relevant  to 
the  DoD,  where  the  applications/systems  to  be  supported  are 
often  unique  or  highly  specialized. 

System  Design 

The  software  scorecard  is  intended  to  examine  the  enabling 
practices  and  capabilities  being  applied  to  a  software  develop¬ 
ment  effort.  It  is  not  intended  to  make  a  detailed  software  en¬ 
gineering  assessment  of  the  code  and  design  itself.  However, 
the  following  key  elements  of  the  design  can  be  assessed  to 
ensure  that  the  design  addresses  reliability  drivers: 

■  System  Architecture 

■  Modular  Design 

■  Data  Architecture 

■  Interface  Design 

■  Fault  Tolerance 

■  Usability 

■  COTS  Selection 

The  scorecard  treats  Usability  as  an  element  integral  to  the 
software's  reliability.  This  is  particularly  true  for  DoD  appli¬ 
cations,  where  a  misunderstanding  by  the  user  or  a  series  of 
small  disruptions  can  endanger  Soldiers  in  combat.  Similarly, 
Soldiers  rely  on  software  prepared  to  handle  Faults  (internal 
and  with  other  interfaced  systems)  with  minimal  disruption. 
The  physically  dispersed  and  austere  locations  in  which  the 
Army  operates  require  that  a  well-designed  Data  Architecture 
consider  the  physical  location  of  data  and  users  and  the  real¬ 
istic  levels  of  communication  between  them. 

Design  for  Reliability  (DFR) 

The  category  of  DFR  elements  endeavors  to  ensure  that 
practices  crucial  to  the  design  of  any  reliable  product  are 
integral  to  the  development  effort.  These  practices  need 
to  be  applied  from  the  very  start  of  development;  program 
risk  should  be  assessed  higher  if  they  are  not.  The  DFR 
elements  consist  of: 

■  Failure  Management 

■  Developmental  Testing 

■  Reliability  Monitoring 

■  System  Reliability  Analysis 

■  Independent  Reviews 

It  is  significant  to  note  that  Developmental  Testing  is  called 
out  as  a  separate  element.  Solid  DFR  requires  that  most  reli¬ 
ability  be  "baked  in"  the  software  and  design  before  customer 
testing.  This  means  that  the  developer  must  expose  failure 
modes  as  early  as  possible.  Each  mode  cannot  be  resolved 
immediately.  However,  a  general  characterization  of  the  failure 
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mode  enables  the  developer  to  plan  when  and  how  it  should 
be  addressed.  Mature  developers  will  reduce  risk  and  costs  by 
conducting  developmental  testing  to  expose,  characterize  and 
prioritize  failure  modes  as  soon  as  possible.  The  subsequent 
customer  test  then  becomes,  as  appropriate,  a  refinement 
stage  to  iron  out  a  minimum  of  issues. 

(Customer)  Testing  and  Acceptance 

It  is  preferable  that  a  developer  bring  a  highly  reliable  product 
to  customer  testing— one  that  requires  minimal  corrections. 
The  reality  is  that  many  software  systems  are  provided  to 
the  customer  with  a  low  level  of  reliability.  This  has  been 
particularly  true  of  many  DoD  applications  that  call  for  a  level 
of  sophistication,  fault  tolerance  and  size  that  are  hard  to 
simulate  in  a  development  environment.  Generally,  new  DoD 
systems  experience  new  failure  modes  and  a  subsequent 
decline  in  reliability  when  they  are  fielded.  Correcting  these 
failures  is  extremely  expensive  and  difficult.  Accordingly,  it 
is  vital  that  customer  acceptance-testing  expose  and  resolve 
as  many  remaining  failure  modes  and  problems  as  possible. 
Further,  the  developer  must  have  the  capabilities  and  tools 
in  place  to  handle  the  new  issues  as  they  arise.  The  numer¬ 
ous  elements  of  (Customer)  Testing  and  Acceptance  seek  to 
address  these  issues.  The  elements  are: 

■  Test  Coverage 

■  Companion  Test  Systems 

■  Test  Depth 

■  Usability  and  Suitability 

■  Scalability 

■  Non-functional  Characteristics 

■  Embedded  Systems  Testing 

■  Failure  Analysis 

■  Software  Code  Analysis  Tools 

Fielding  and  Sustainment 

The  utility  of  a  software  reliability  scorecard  is  less  appar¬ 
ent  when  a  project  has  reached  the  Fielding  or  Sustainment 
stage.  However,  the  fielding  and  maintenance  of  existing 
software  requires  significant  resources.  For  large  applica¬ 
tions,  the  financial  cost  is  substantial.  The  elements  of  this 
category  assess  how  well  a  project  has  prepared  for  these 
stages  of  the  life  cycle.  It  is  recommended  that  the  prior  ele¬ 
ments  of  the  scorecard  also  be  assessed  at  this  time.  They 
provide  insight  into  the  quality  and  character  of  the  software 
on  hand.  A  mature  developer  will,  for  example,  know  what 
issues  (e.g.,  failure  modes)  remain  unresolved.  The  elements 
addressed  regarding  Fielding  and  Sustainment  are: 

■  Software  Maintenance 

■  Field  Support 

■  Documentation  for  Sustainment 

■  Dependencies  and  Interoperability 

■  Sustainment  Testing 

■  Training 

■  Distribution 

■  Continuity  of  Operations  Plan  (COOP) 
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In  the  earlier  stages  of  software  development,  scorecard  users 
are  encouraged  to  consider  and  assess  the  Fielding  and  Sus¬ 
tainment  elements.  Though  these  activities  may  not  be  due 
yet,  a  proactive  developer  will  start  working  on  them  in  parallel 
with  earlier  activities.  They  can  serve  as  "leading  indicators" 
of  a  low-risk  project. 

Summary  and  Conclusion 

AMSAA  developed  the  software  scorecard  to  complement  its 
existing  hardware  reliability  portfolio  and  facilitate  an  increas¬ 
ing  software  reliability  workload.  Its  primary  use  is  to  assess 
the  reliability  practices  of  individual  software  projects  and  to 
focus  more  detailed  analysis  and  work  on  the  areas  of  most 
concern.  It  is  a  particularly  useful  approach  when  "triaging" 
ongoing  software  projects  to  identify  where  to  focus  analysis 
and  support.  AMSAA  applies  the  scorecard  methodology  to 
provide  Army  and  other  DoD  programs  an  independent  as¬ 
sessment  of  the  project's  level  of  risk.  However,  the  instrument 
can  be  applied  by  the  customer  or  as  a  self-assessment  tool 
by  the  developer.  Valuable  ideas  and  feedback  from  industry 
partners  and  other  DoD  organizations  have  been  used  to  im¬ 
prove  this  tool  for  more  general  use. 

The  software  scorecard  is  available  at  no  cost  to  any  U.S.  gov¬ 
ernment  agency  and  its  contractors.  Information  to  request  a 
copy  is  available  at  www.amsaa.army.mil/ReliabilityTechnol- 
ogy/RelTools.html.  & 


The  authors  can  be  contacted  at  timothy.g.pohland.civ@mail.mil  and 
david.g.bernreuther.civ@mail.mil. 
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n  the  May-June  2013  issue  of  Defense  AT&L  magazine,  Under  Secretary  of  Defense  for  Acqui¬ 
sition,  Technology  and  Logistics  Frank  Kendall  authored  an  article  titled  "The  Original  Better 
Buying  Power— David  Packard  Acquisition  Rules  1971."  Packard's  fifth  acquisition  rule  was 
"fly  before  you  buy”— the  underpinnings  of  test  and  evaluation  (T&E).  I  thought  it  might  be 
interesting  to  look  at  another  challenge  that  then  Deputy  Secretary  of  Defense  David  Pack¬ 
ard  confronted  in  1971:  what  to  do  about  test  and  evaluation  in  the  Department  of  Defense 
(DoD).  The  original  Director  for  Test  and  Evaluation  created  by  David  Packard  was  a  DT&E 
with  broad  responsibilities  for  all  T&E  matters  in  the  DoD.  Today's  DT&E  is  focused  only  on 


Hutchison  is  the  principal  deputy  for  developmental  test  and  evaluation  in  the  Office  of  the  Secretary  of  Defense. 
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Developmental  Test  and  Evaluation.  This  article  provides  a 
brief  snapshot  into  the  challenges  involved  in  forming  the  first 
DT&E  office  in  the  Pentagon  and  its  evolution  into  the  DT&E 
office  today. 

In  July  1970,  President  Nixon's  Blue  Ribbon  Defense  Panel 
(BRDP)  reported  its  findings.  The  "Fitzhugh  Commission,"  as 
it  is  sometimes  called,  in  recognition  of  its  chairman,  Gilbert 
W.  Fitzhugh,  had  been  appointed  1  year  earlier  with  the  broad 
charter  to  report  and  make  recommendations  on  the  organiza¬ 
tion  and  management  of  the  DoD,  its  research  and  develop¬ 
ment  efforts,  and  its  procurement  policies  and  practices.  The 
final  report  prominently  featured  a  discussion  of  operational 
test  and  evaluation,  and  the  panel's  recommendations  had 
significant  influence  on  how  David  Packard  would  shape  the 
role  of  T&E  to  support  defense  acquisition.  (The  BRDP  is  avail¬ 
able  at  http://www.dtic.mil/dtic/tr/fulltext/u2/a013261.pdf .) 


responsibilities  for  any  evaluation  function  must  be  exercised 
independently.  When  they  are  subordinated  to  or  combined 
with  responsibilities  for  the  development  of  the  item  or  subject 
being  evaluated,  the  requisite  objectivity  is  seriously  jeopar¬ 
dized."  The  BRDP  also  considered  evaluation  within  a  much 
broader  construct  and  recommended  that  the  Secretary  create 
a  position  of  "Deputy  Secretary  of  Defense  for  Evaluation"  sup¬ 
ported  by  three  Assistant  Secretaries:  Comptroller,  Program 
and  Force  Analysis,  and  Test  and  Evaluation.  The  BRDP  fur¬ 
ther  recommended  that  the  Secretary  create  a  Defense  Test 
Agency  "to  perform  the  functions  of  overview  of  all  Defense 
test  and  evaluation,  designing  or  reviewing  of  designs  for  test, 
monitoring  and  evaluation  of  the  entire  Defense  test  program, 
and  conducting  tests  and  evaluations  as  required,  with  par¬ 
ticular  emphasis  on  operational  testing,  and  on  systems  and 
equipments  which  span  Service  lines." 


The  BRDP  had  three  main  findings  relative  to  the  conduct  of 
T&E:  "functional  testing"  (we  use  the  term  "developmental 
testing"  today)  is  fundamentally  sound;  operational  testing  is 
generally  inadequate,  and  the  Services  do  not  conduct  enough 
joint  test  and  evaluation.  Regarding  "functional"  testing,  the 
BRDP  wrote: 

By  and  large,  functional  testing  in  and  for  the  Department  of 
Defense  appears  to  be  well  understood  and  faithfully  executed. 
Serious  policy  deficiencies  are  not  apparent,  and  such  failures 
in  functional  testing  as  occur  can  be  primarily  attributed  to  lack 
of  technical  competence,  oversight,  or  procedural  breakdowns. 
Functional  testing  is  not  considered  to  be  a  major  problem  area. 

Pertaining  to  operational  test  and  evaluation,  however,  the 
BRDP  found  that  "Operational  test  and  evaluation  has  been 
too  infrequent,  poorly  designed  and  executed,  and  generally 


Packard  promptly  began  acting  on  the  BRDP  recommenda¬ 
tions.  In  the  first  8  months  of  1971,  he  signed  three  memoranda 
that  made  sweeping  changes  to  the  role  of  test  and  evaluation 
in  support  of  defense  acquisition. 

In  the  first  memorandum,  "Conduct  of  Operational  Test  and 
Evaluation,"  dated  Feb.  11, 1971,  Packard  wrote, "...  a  number 
of  specific  actions  must  be  taken  to  put  our  house  in  order  in 
this  very  important  aspect  of  the  weapon  system  acquisition 
process."  Packard  directed  three  specific  actions: 

■  Services  will  restructure  their  organizations  for  OT&E  to  be 
"separate  and  distinct  from  the  developing  command"  and 
report  the  results  of  its  test  and  evaluation  efforts  "directly 
to  the  Chief  of  the  Service." 

■  OSD  will  establish  "a  Deputy  Director  for  Test  and 
Evaluation  within  the  Office  of  Director  of  Defense  Re¬ 
search  and  Engineering  (ODDR&E)  with  across-the-board 


inadequate."  The  panel's  recommendations  regarding  T&E, 
and  thus  the  actions  taken  by  DoD  to  implement  those  recom¬ 
mendations,  essentially  focused  on  correcting  deficiencies  in 
operational  test  and  evaluation  (OT&E),  and  have  driven  the 
primary  focus  in  defense  acquisition  for  the  past  40  years. 

The  BRDP  also  highlighted  the  lack  of  OT&E  oversight  in 
OSD  as  a  "glaring  deficiency"  and  reported  that  "In  connec¬ 
tion  with  test  and  evaluation,  it  should  be  emphasized  that 


responsibilities  for  OSD  in  test  and  evaluation  matters.  This 
office  will  review  and  approve  test  and  evaluation  plans 
prepared  by  the  Services  and  will  provide  an  assessment 
of  results  obtained." 

■  Services  will  conduct  more  joint  operational  test  and 
evaluation. 

In  the  Feb.  11, 1971,  memo,  Packard  asked  the  Service  Secre¬ 
taries  to  advise  him  of  their  plans  before  March  31,  and  to  "be 
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prepared  to  have  the  new  structure  in  effect  by  the  end  of  this 
fiscal  year.”  Prior  to  1976,  the  fiscal  year  began  on  July  1  and 
ended  on  June  30;  in  other  words,  Packard  gave  the  DoD  just 
4  months  to  put  its  house  in  order!  Clearly,  this  was  a  high 
priority  for  the  Deputy  Secretary. 


Evaluation  would  be  responsible  for  “initiating  and  coordinat¬ 
ing  appropriate  joint  testing;  overseeing  the  test  and  evaluation 
of  foreign  systems  for  possible  DoD  use;  and  administering 
for  OSD  its  responsibilities  for  the  national  and  major  Service 
test  ranges.” 


The  second  of  Packard's  memoranda,  dated  April  21, 1971,  ce¬ 
mented  the  role  of  the  Deputy  Director  for  T&E  in  the  Defense 
System  Acquisition  Review  Council  (DSARC)  (the  DSARC  was 
the  forerunner  of  the  modern  Defense  Acquisition  Board). 


Packard  needed  someone  to  lead  the  new  organization  and 
make  his  vision  reality.  On  June  7, 1971,  Packard  appointed 
retired  Lt.  Gen.  Alfred  Dodd  Starbird  to  be  the  first  Deputy 
Director  for  Test  and  Evaluation.  Lt.  Gen.  Starbird  set  out  to 


Packard  directed  the  Military  Departments  to  provide  their 
assessment  of  test  results  to  the  DDR&E  prior  to  the  DSARC 
Milestone  III  production  decision.  The  Deputy  Director  for 
Test  and  Evaluation  (DDT&E)  would  review  those  results  and 
provide  his  independent  recommendations  at  the  Milestone 
III  meeting. 

Packard  also  quickly  began  honing  the  new  processes  as  they 
were  implemented.  On  Aug.  3, 1971,  Packard  wrote,  “Based  on 
experience  to  date  resulting  from  [the  previous  two  memos], 
I  consider  certain  further  clarification  and  instruction  is  re¬ 
quired.”  In  this  third  memorandum,  “Test  and  Evaluation  in 
System  Acquisition  Process,”  Packard  described  in  more  de¬ 
tail  requirements  for  conduct  of  adequate  OT&E.  Addition¬ 
ally,  Packard  assigned  these  additional  responsibilities  to  the 
Deputy  Director  for  Test  and  Evaluation: 

■  “be  responsible  within  OSD  for  reviewing  test  and  evaluation 
policies  and  procedures  and  recommendingto  me  changes 
as  appropriate; 

■  “monitor  closely  test  and  evaluation  conducted  by  the  Ser¬ 
vices  for  DSARC  programs,  and  such  other  programs  as  he 
believes  necessary,  throughout  the  entire  testing  cycle; 

■  “report  to  the  DSARC  and  directly  to  me  at  DSARC  Mile¬ 
stones!  and  II  his  assessment  as  to  the  adequacy  of  the  list 
of  critical  issues  and  problems  to  be  attacked  by  test  and 
evaluation  and  the  schedule  of  test  milestones; 

■  and  “report  at  Milestone  III  to  the  DSARC  and  directly  to  me 
his  independent  recommendation.” 

The  Deputy  Secretary  went  on  to  require  that  “Service  test 
plans  and  test  results  be  made  available  at  his  request  as  early 
as  developed.”  Additionally,  the  Deputy  Director  for  Test  and 


build  the  new  DDT&E  organization.  He  had  three  assistant 
directors  with  responsibilities  for  Tactical  Systems  T&E,  Stra¬ 
tegic  and  Support  Systems  T&E,  and  Test  Resources.  During 
his  tenure,  Starbird  recommended  the  Defense  Science  Board 
(DSB)  examine  test  and  evaluation  in  the  department.  The  DSB 
published  the  first  report  on  T&E  in  April  1974;  it  would  be  the 
first  of  six  reports  the  DSB  would  issue  on  T&E  over  the  next 
40  years.  Starbird  also  initiated  the  publication  of  the  first  DoD 
Directive  on  test  and  evaluation:  DoD  Directive  5000.3,  Test 
and  Evaluation,  was  published  on  Jan.  19, 1973.  The  5000.3 
continued  in  effect,  with  four  updates,  until  1991,  when  the 
February  1991  issuance  of  DoD  Directive  5000.1  canceled  the 
5000.3  and  T&E  became  Part  8  of  DoD  Instruction  5000.2. 
Starbird  also  was  able  to  insert  into  the  defense  budget  a  new 
appropriation  for  the  deputy  director  of  T&E.  The  Department 
of  Defense  Appropriation  Act  for  FY 1973,  Public  Law  92-570, 
dated  Oct.  26, 1972,  included  the  following  language: 

DIRECTOR  OF  TEST  AND  EVALUATION,  DEFENSE 

For  expenses,  not  otherwise  provided  for,  of  independent  ac¬ 
tivities  of  the  Director  of  Defense  Test  and  Evaluation  in  the 
direction  and  supervision  of  test  and  evaluation,  including  initial 
operational  testing  and  evaluation;  and  performance  of  joint 
testing  and  evaluation;  and  administrative  expenses  in  connec¬ 
tion  therewith,  $27,000,000,  to  remain  available  for  obligation 
until  June  30,1974. 

Thus  did  David  Packard  implement  the  recommendations  of 
the  BRDP  Although  there  would  not  be  a  Deputy  Secretary  for 
Evaluation,  or  an  Assistant  Secretary  for  T&E,  he  did  create  the 
“defense  test  agency”  in  the  form  of  the  Office  of  the  Deputy 
Director  for  Test  and  Evaluation  in  the  Office  of  the  Director 
of  Defense  Research  and  Engineering.  The  Deputy  Director 
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for  T&E  was  responsible  for  all  matters  involving  T&E  almost 
exactly  as  the  BRDP  recommended.  Importantly,  Packard  also 
ensured  the  Deputy  Director  for  T&E  provided  his  independent 
assessment  at  the  DSARC.  However,  independence  came  to 
be  another  issue  entirely,  and  the  T&E  organization  Packard 
put  in  place  would  not  endure. 

Concerns  about  OT&E  continued  to  consume  attention  in  the 
Pentagon  and  especially  on  the  Hill.  Likewise,  there  was  an 
overwhelming  sense  that  assignment  of  the  Deputy  Director 
for  Test  and  Evaluation  under  the  DDR&E  posed  a  conflict  of 
interest.  The  DDR&E  was  the  DoD's  chief  acquisition  official 
at  the  time,  which  violated  the  BRDP's  premise  that  when  re¬ 
sponsibilities  for  evaluation  "are  subordinated  to  or  combined 
with  responsibilities  for  the  development  of  the  item  or  subject 
being  evaluated,  the  requisite  objectivity  is  seriously  jeopar¬ 
dized."  I  doubt  that  Starbird,  or  any  of  the  deputies  after  him, 
felt  that  way,  but  regardless,  in  September  1983,  Congress 
established  a  presidential^  appointed,  Senate-confirmed  posi¬ 
tion  of  Director,  Operational  Test  and  Evaluation,  independent 
of  the  acquisition  authority  and  reporting  directly  to  the  Secre¬ 
tary  of  Defense.  The  position  of  Deputy  Director  for  Test  and 
Evaluation  was  renamed  Deputy  Director  for  Developmental 
Test  and  Evaluation.  Another  16  years  later,  on  June  7, 1999— 
or  28  years  after  Packard  created  it— Secretary  of  Defense 
William  Cohen  disestablished  the  test  office  within  what  had 
become  the  Office  of  the  Undersecretary  of  Defense  for  Ac¬ 
quisition  and  Technology.  During  those  nearly  3  decades,  all 


emphasis  in  T&E  in  the  department  continued  to  be  on  opera¬ 
tional  testing,  and  Cohen's  decision  was  intended  specifically 
to  strengthen  the  Office  of  the  DOT&E;  however,  it  virtually 
eliminated  oversight  of  developmental  test  and  evaluation. 
Congress  would  reverse  this  10  years  later.  How  that  came  to 
pass  is  another  story. 

As  a  member  of  the  DT&E  organization  in  OSD  today,  my  view 
of  test  and  evaluation  is  obviously  biased.  But  with  15  years' 
experience  in  the  T&E  community,  most  of  which  has  involved 
operational  testing  in  some  capacity,  I  have  come  to  believe 
that  the  department's  decades-long  emphasis  on  OT&E  in  the 
acquisition  life  cycle  took  our  eye  off  the  target.  Acquisition 
success  is  not  about  passing  OT&E  and  getting  that  full-rate 
production  decision;  success  occurs  when  we  have  properly 
set  the  conditions  to  begin  production.  The  key  to  improv¬ 
ing  acquisition  outcomes  is  to  get  the  development  right,  and 
verify  it  through  rigorous  developmental  test  and  evaluation. 
We  have  to  do  a  better  job  helping  programs  get  to  Milestone 
C;  that's  the  target  we  must  aim  for.  In  T&E,  we  have  to  shift 
left  if  we  are  going  to  "fly  before  you  buy."  Packard  had  a  good 
plan  back  in  1971,  but  we  lost  track  of  the  target.  Given  his  un¬ 
precedented  (and  unmatched)  level  of  involvement  in  shaping 
the  course  for  T&E  in  the  department,  I  think  one  could  make 
a  convincing  argument  that  David  Packard  was,  himself,  the 
original  DT&E.  & 


The  author  can  be  contacted  at  steven.j.hutchison.civ@mail.mil. 
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The  Program  Managers  e-Tool  Kit  provides 
the  program  management  resources 
of  the  popular  print  Program 
Managers  Tool  Kit  in  a  dynamic 
Web-based  format. 

The  e-Tool  Kit  features: 

■  Continual  content  updates 

■  Live  policy  links 

■  Links  to  informative  ACQuipedia  articles 
and  related  communities  of  practice. 


Visit  https://pmtoolkit.dau.mil/ 
today  to  explore  this  convenient  tool! 
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n  the  summer  of  2012,  Heidi  Shyu,  the  recently 
confirmed  Army  Acquisition  Executive,  directed 
each  Program  Executive  Office  (PEO)  to  develop 
a  30-year  strategic  plan.  The  plan  was  to  focus  on 
linking  science  and  technology  (S&T)  projects  to 
programs  of  record,  as  well  as  modernization  of  existing 
fielded  equipment.  Each  Army  PEO  developed  its  own 
plan,  which  mapped  its  programs  of  record  to  capability 
gaps  and  known  S&T  efforts  meant  to  close  those  gaps. 
These  plans  were  to  address  challenges  leadership  faced 
in  obtaining  in-depth  information  to  support  fact-based 
decision  making. 

Army  leadership  constantly  must  make  decisions  related  to  prioritizing  fund¬ 
ing,  scheduling  and  materiel  acquisition;  and  these  decisions  are  becoming  more 
complex  given  the  current  and  projected  fiscal  challenges.  Second-  and  third- 
order  effects  of  decisions  become  increasingly  important,  and  these  plans  were 
intended  to  provide  insight  into  those  effects. 

In  parallel,  Deputy  Chief  of  Staff  of  the  Army  G8  initiated  development  of  the 
modernization  plans  for  existing  equipment  known  as  LIRA,  or  Long-Range  Invest¬ 
ment  Requirements  Analysis.  These  plans  were  meant  to  answer  the  question 
"how  much  of  tomorrow's  dollars  are  we  committing  by  spending  dollars  today?" 

Subsequently,  the  Research,  Development  and  Engineering  Command  (RDE- 
COM)  and  its  subordinate  centers  and  laboratories,  under  direction  of  the  Deputy 
Assistant  Secretary  of  the  Army  for  Research  and  Technology,  developed  30- 
year  roadmaps  linking  their  technology  initiatives  to  capability  gaps  and  known 
programs  of  record.  Again,  the  purpose  of  these  plans  was  to  support  fact-based 
decisions  by  providing  a  longer  range  look  at  the  consequences  and  effects  of 
those  decisions. 


Matrisciano  has  more  than  20  years  of  research  and  development  program  experience  within  the 
Army  and  is  currently  the  Research  &  Development  Program  Coordinator  at  the  Program  Executive 
Office  for  Ammunition,  Picatinny  Arsenal,  N.J.  He  is  responsible  for  defining  the  PEO's  near-,  mid-  and 
far-term  research  and  development  priorities,  including  the  30-year  strategy.  He  is  collaborating  with 
counterparts  in  other  PEOs  and  R&D  organizations  in  developing  an  integrated  strategy. 
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It  is  important  that  all  stakeholders 
are  aligned  in  the  same  direction, 
understand  the  current  version  of  the 
goal  (the  “big  picture"  and  their  piece 


of  it)  and  leverage  each  other's  efforts 

i  i  *  ' 

to  achieve  that  goal. 


& 


* 
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Although  diligent  and  fervent  work  went  into  each  of  the  plans, 
they  were  all  developed  semi-independently.  Limited  coordina¬ 
tion  among  plan  authors  provided  some  synergy  among  the 
plans,  but  they  remained  largely  independent.  And  currently 
there  is  no  clear  path  to  integrate  these  plans. 

During  the  development  of  these  plans,  the  predominant 
questions  from  participants  were,  "Why  are  we  doing  this, 
especially  given  current  funding  limitations?"  and  "How  can 
we  know  what  life  will  be  like  30  years  from  now?" 

Both  are  valid  questions.  After  all,  who  30  years  ago  could  have 
imagined  all  the  products  we  use  today?  Regardless,  there 
are  significant  benefits  to  developing  a  30-year  strategic  plan, 
even  if  we  cannot  accurately  forecast  life  in  2044.  It  all  begins 
with  the  value  of  planning. 

Benefits  of  Planning 

As  acquisition  professionals,  we  all  are  taught  that  a  good 
plan  (or  any  plan  for  that  matter)  is  a  valuable  tool.  It  pro¬ 
vides  a  barometer  from  which  to  measure  our  progress  and 
success,  helping  us  maintain  control  of  our  activity.  Project 
managers  typically  plan  their  programs  before  they  start  to 
execute  them.  As  most  of  us  have  experienced,  however, 
forecasting  the  next  few  years  of  a  program  is  difficult,  and 
some  programs  fail  or  do  not  meet  their  goals  fully  despite 
a  well-thought-out  plan. 

As  a  result,  many  of  us  have  learned  that  the  plan  is  not  the 
panacea,  not  the  end-all  and  be-all,  but  only  the  first  step  in  the 
program  management  process.  Strict  diligence  in  monitoring 
performance  and  risk,  and  adjusting  as  necessary,  are  keys  to 
success.  And  so  it  is  with  our  30-year  strategic  plans.  They  are 
a  best  guess  based  on  what  we  know  today,  but  probably  are 
not  very  accurate.  Nevertheless,  they  provide  an  orientation 
from  which  to  proceed,  and  a  rationale  for  why  we  are  going 
that  way.  This  allows  all  stakeholders  to  "be  on  the  same  page" 
and  work  toward  a  common  goal. 

When  completed  and  integrated,  the  30-year  plan  should 
show  the  "big  picture,"  linking  all  capability  gaps  to  S&T  ac¬ 
tivity,  all  S&T  activity  to  fielding  of  materiel,  and  every  fielding 


to  operations,  support  and  eventual  disposal  and/or  replace¬ 
ment.  An  important  benefit  is  minimization  of  the  Army's  de¬ 
militarization  stockpile  through  greater  emphasis  on  strategic 
planning  at  the  materiel  level.  The  finished  product  not  only 
shows  what  needs  to  be  done  throughout  the  life  cycle,  but 
how  much  it  will  cost  and  when  it  should  happen.  Any  breaks 
in  the  linkage  are  highlighted  and  addressed  by  informed  lead¬ 
ership  decisions,  either  by  strengtheningthe  links  or  eliminat¬ 
ing  them  altogether. 

Integrated  Planning 

To  be  effective,  these  30-year  strategic  plans  must  be  inte¬ 
grated  and  include  the  same  diligent  monitoring  that  is  applied 
to  acquisition  programs.  To  achieve  integration,  the  individual 
PEO  and  RDECOM  plans  must  be  combined  to  become  the 
cohesive  30-year  strategic  plan.  This  will  ensure  that  all  inter¬ 
faces  and  relationships  among  systems  and  programs  are  con¬ 
sidered.  Integration  of  the  individual  plans  is  currently  evolving 
at  the  "grass  roots"  level.  PEOs  and  Research  Development 
and  Engineering  Centers  (RDECs)  are  coordinating  with  each 
other  on  mutual  touch  points  to  ensure  that  their  efforts  are 
useful  and  efficient.  However,  this  process  is  not  fully  effective. 
While  there  is  some  integration  in  some  areas,  other  areas 
are  being  missed.  In  other  words,  integration  is  "ad  hoc."  To 
streamline  this  process,  the  standard  acquisition  Integrated 
Product  Team,  or  IPT,  model  should  be  used.  As  applied  here, 
the  members  of  the  IPT  represent  each  of  the  PEOs,  RDECOM, 
TRADOC,  and  Army  headquarters  staff  (and  others  as  ap¬ 
propriate).  An  IPT  Lead  would  manage  the  integrated  planning 
effort  to  ensure  that  roles  and  responsibilities  are  defined  and 
the  common  goal  of  an  integrated  plan  is  achieved. 

Maintaining  the  Integrated  Plan 

Once  the  plan  is  completed  and  baselined,  adequate  moni¬ 
toring  is  vital  to  avoid  having  it  become  "shelf  ware."  The 
process  for  monitoring  the  plans  also  should  mirror  the  IPT 
model  described  above,  with  the  IPT  Lead  running  regularly 
scheduled  formal  discussions  (i.e.,  quarterly,  semi-annually, 
etc.)  to  ensure  the  entire  team  remains  aligned,  manages  risk, 
communicates  status  and  updates  the  plan  as  needed.  When 
assumptions  become  reality,  the  plan  is  updated.  When  "near 
term"  planned  activities  come  to  fruition,  "out  years"  are  added 
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so  it  is  a  rolling  30-year  plan.  It  is  important  that  all  stakehold¬ 
ers  are  aligned  in  the  same  direction,  understand  the  current 
version  of  the  goal  (the  "big  picture"  and  their  piece  of  it)  and 
leverage  each  other's  efforts  to  achieve  that  goal. 

Maintenance  of  the  plan  also  includes  regular  interaction  with 
senior  leadership  to  communicate  plan  contents  and  status, 
as  well  as  obtain  feedback  on  any  required  adjustments  based 
on  changing  priorities  and/or  updated  strategy.  Since  its  over¬ 
all  purpose  is  to  inform  leadership  decisions,  the  plan  must 
become  a  standard  "front  and  center"  fixture  in  the  decision¬ 
making  process.  For  the  plan  to  be  useful,  senior  leaders  must 
routinely  consider  the  information  it  provides. 

Why  Now? 

Another  common  question  is,  "The  Army  has  been  around  a 
long  time  and  we've  never  had  a  30-year  plan,  so  why  do  we 
need  one  now?" 

One  could  also  ask  why— though  the  Army  continues  to  field 
some  of  the  best  equipment  in  the  world— programs  still  en¬ 
counter  roadblocks  or  dead  ends.  How  many  of  those  fielded 
items  could  have  been  fielded  sooner  and  at  a  lower  life-cycle 
cost?  How  many  overlapping  capabilities  exist?  How  many 
technologies  did  not  get  fielded  even  though  they  achieved 
technical  success?  Why  are  there  so  many  items  in  our  demili¬ 
tarization  account?  Now  more  than  ever,  the  answers  to  these 


questions  have  strategic  relevance,  but  they  remain  elusive 
without  an  integrated  long-term  plan  from  which  to  acquire 
this  knowledge. 

No  doubt  we  can  collectively  work  more  efficiently  while  re¬ 
maining  effective,  and  the  integrated  long-term  plan— and  on¬ 
going  maintenance  of  the  plan— is  essential  for  that  to  happen. 
It  provides  a  mechanism  to  ensure  that  our  efforts  are  comple¬ 
mentary  and  neither  duplicative  nor  wasted  by  showing  how 
they  fit  into  the  long-term  strategy  while  highlighting  second- 
and  third-order  effects.  In  other  words,  it  provides  more  and 
better  information  to  feed  fact-based  leadership  decisions. 

Although  long-term  plans  like  this  30-year  strategic  plan  are 
far  from  perfect,  they  provide  the  required  baseline  from  which 
to  operate  and  support  informed  decisions.  In  the  current  cli¬ 
mate  of  fiscal  uncertainties,  long-term  planning  will  help  pro¬ 
vide  more  "bang  for  the  buck"  by  guiding  informed  investment 
decisions  and  identifying  the  second-  and  third-order  effects. 
The  key  to  effective  and  efficient  fielding  of  equipment  to  the 
warfighter  is  active  leadership  in  developing,  monitoring  and 
maintaining  the  collective  plan.  With  budgets  declining,  and 
no  sign  of  recovering,  we  owe  our  ultimate  customers— the 
warfighter  and  the  taxpayer— the  best  we  can  deliver  in  the 
most  efficient  manner  possible.  & 

The  author  can  be  reached  at  vincent.r.matrisciano.civ@mail.mil. 


Where  Can  You  Get 
the  Latest  on  the 

Better  Buying  Power 

Initiatives? 


■  BBP  Gateway  (https://dap.dau.mil/bbp)  is  your  source  for  the 
latest  information,  guidance,  and  directives  on  better  buying 
power  in  defense  acquisition 


■  BBP  Public  Site  (https://acc.dau.mil/bbp)  is  your  forum  to  share 
BBP  knowledge  and  experience 
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Promoting  Effective 
Competition 


Competition  throughout  the  life  cycle  of  an  acquisition  program  not  only  is  possible,  it  is 
alive  and  well  in  the  Joint  Light  Tactical  Vehicle  (JLTV)  program.  This  article  focuses  on 
the  JLTV  program  and  the  Better  Buying  Power  2.0  (BBP  2.0)  effort  to  “Promote  Effec¬ 
tive  Competition.”  Although  the  JLTV  program  began  before  BBP  2.0,  the  overarching 
concept  of  competition  throughout  the  life  cycle  of  an  acquisition  program  is  not  new 
and  remains  a  key  component  of  the  Federal  Acquisition  Regulation. 

Program  Overview 

Development  of  the  JLTV  is  a  joint  program  to  augment  the  High  Mobility  Multipurpose  Wheeled  Vehicle 
(HMMWV)  fleet  currently  in  service  with  the  U.S.  Army  and  U.S.  Marine  Corps.  Forthe  Marine  Corps,  it  is  designed 
to  replace  HMMWVs  only  for  the  most  demanding  mission  profiles,  and  for  the  Army,  to  replace  "approximately 
1/3  of  the  light  wheeled  vehicle  fleet  by  2040."  The  JLTV  family  of  vehicles  will  provide  additional  survivability,  a 
greater  payload,  and  responsive,  well-integrated  command  and  control.  Changes  in  contemporary  threats,  coupled 
with  the  inability  of  the  HMMWV  platform  to  accommodate  the  magnitude  of  change  needed  to  meet  the  new 
requirements,  made  the  JLTV  program  necessary. 


Mills  is  a  former  program  manager  from  Northrop  Grumman ,  Inc.  He  currently  serves  as  a  professor  of  program  management  and  informa¬ 
tion  technology  at  the  Defense  Acquisition  University. 
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“Do  I  spend  precious  time  and  resources  to  compete 
for  the  EMD  phase  of  the  JLTV  program  even  though  I 
Vst  lost  the  TD-pha^^competition?"  m^cpnclusion 
many  outside  observerswould  come  to  is  “ftoKL 


The  JLTV  program  requirements  were  approved  by  the  Joint 
Requirements  Oversight  Council  in  November  2006  and  des¬ 
ignated  an  Acquisition  Category  ID  program  with  the  Army 
as  the  lead.  This  program  has  experienced  some  significant 
challenges,  including  Technology  Development  (TD)  phase 
protests  from  two  offerors  and  a  threatened  program  termi¬ 
nation  at  the  end  of  the  phase.  In  September  2011,  the  Senate 
Appropriations  Defense  Subcommittee  recommended  can¬ 
celing  the  JLTV  due  to  rising  program  costs  and  continuous 
changes  in  requirements.  The  subcommittee  endorsed  the 
idea  of  changing  the  Modernized  Expanded  Capacity  Vehicle 
(MECV)  program  (HMMWV  Recapitalization)  from  a  stop¬ 
gap  effort  to  a  full  solution  to  meet  the  vehicle  requirements 
of  the  Army  and  Marine  Corps  in  place  of  the  JLTV. 

As  a  full  solution,  MECV  funding  would  have  had  to  increase 
significantly.  Once  the  subcommittee  recommendation  was 
released,  the  Army  and  Marine  Corps  finalized  the  JLTV 
requirements  and  established  a  cost  cap  of  $250,000  per 
vehicle.  These  new  program  requirements  were  codified  in 
the  Engineering,  Manufacturing  and  Development  (EMD) 
Phase  Request  for  Proposal  (RFP)  and  proved  effective  in 
reversing  the  recommendation  of  the  subcommittee.  The 
MECV  program  was  canceled  in  January  2012.  The  re¬ 
structured  JLTV  program  is  in  the  EMD  phase  with  three 
competing  industry  partners  (Lockheed  Martin,  Oshkosh 
Corporation  and  AM  General).  Low-rate  initial  production 
is  planned  for  2015,  with  full-rate  production  scheduled  for 
2018.  The  Army  intends  to  purchase  49,909  vehicles,  and 
the  Marine  Corps,  5,500. 

As  noted  above,  BBP  2.0  clearly  focuses  on  the  importance 
and  value  to  the  government  of  both  creating  and  maintain¬ 
ing  a  competitive  environment.  The  JLTV  Joint  Program  Of¬ 
fice  used  requests  for  information,  industry  days,  and  draft 
RFPs  to  communicate  program  requirements  effectively  to 
industry  and  to  demonstrate  the  government's  commitment 
to  the  viability  of  the  JLTV  program. 


understanding  of  future  needs.  The  JLTV  acquisition  strat¬ 
egy  and  source  selection  plan  served  as  a  framework  for 
industry  to  conduct  effective  strategic  planning  and  enabled 
industry  to  compete  for  both  TD  and  EMD  contracts.  The 
EMD-phase  RFP  focused  on  "full  and  open"  competition 
and  contained  language  that  specifically  allowed  offerors 
who  were  rejected  in  the  TD  phase  to  compete  for  EMD- 
phase  contracts. 

This  component  of  the  JLTV  acquisition/source  selection 
strategy  created  an  interesting  opportunity  for  industry 
teams  to  ponder:  "Do  I  spend  precious  time  and  resources 
to  compete  for  the  EMD  phase  of  the  JLTV  program  even 
though  I  just  lost  the  TD-phase  competition?"  The  conclusion 
many  outside  observers  would  come  to  is  "No!"  Significant 
time  and  money  would  be  required,  and  convincing  senior 
industry  leaders  to  assume  that  level  of  risk  would  be  difficult. 
Yet,  in  the  case  of  the  JLTV,  two  of  the  three  teams  selected 
for  the  EMD  phase  were  teams  that  had  been  rejected  in 
the  TD  phase. 

This  occurrence  generates  many  questions.  What  motivated 
these  industry  teams  to  compete  for  the  EMD  phase?  What 
is  the  real  story  behind  the  firm's  success  in  the  EMD  phase 
competition?  Is  this  a  David  and  Goliath  story?  One  of  the 
industry  partners  rejected  for  the  TD  phase,  but  selected  for 
the  EMD  phase  was  Oshkosh  Corporation.  Part  of  their  story 
revolves  around  leveraging  other  opportunities: 

"Finally,  Oshkosh,  which  had  its  eye  on  the  JLTV  program 
since  it  lost  out  on  a  technology  development  contract  in 
2008,  is  offering  a  variant  of  its  Light  Combat  Tactical  All- 
Terrain  Vehicle  (L-ATV),"  wrote  Kate  Brannen  ("Competition 
upended  in  JLTV  program")  in  the  March  31,  2012,  Marine 
Corps  Times.  "Oshkosh  said  it  has  been  able  to  build  off  of  les¬ 
sons  learned  from  its  MRAP-ATV  (M-ATV)  effort,  which  was 
designed  to  meet  an  urgent  need  in  Afghanistan  for  a  lighter 
mine-resistant,  ambush-protected  vehicle."  Brannen  added. 


In  the  case  of  the  JLTV,  the  use  of  these  tools  provided  clear 
commitment  to  the  JLTV  program  and  motivated  the  three 
industry  teams— Lockheed  Martin,  BAE  and  General  Tactical 
Vehicles— each  to  have  completed  a  working  JLTV  prototype 
before  release  of  the  final  RFP  during  the  TD  phase.  This  type 
of  response  from  the  offerors  has  the  potential  to  reduce 
overall  program  technical  risk  as  well  as  program  cost. 

Government  acquisition  programs'  overall  risk  is  reduced 
when  the  government  provides  industry  with  a  clear 


Oshkosh's  efforts  on  the  M-ATV  provided  significant  value 
for  their  EMD-phase  offering.  They  not  only  leveraged  les¬ 
sons  learned  in  Afghanistan,  but  could  claim  recent  and  rel¬ 
evant  past  performance  in  fielding  tactical,  wheeled  vehicles. 
A  quick  review  of  the  Oshkosh  EMD-phase  proposal  reveals 
that  past  performance  was  the  second-highest  rated  factor 
after  technical.  This  is  a  major  change  to  relative  importance 
when  compared  with  the  TD-phase  request;  in  the  TD  phase, 
past  performance  was  much  less  important  when  compared 
with  the  other  factors. 
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The  other  team  that  did  not  win  a  TD-phase  contract,  but  was 
successful  in  the  EMD-phase  competition  was  AM  General, 
manufacturer  of  the  HMMWV.  With  diminishing  HMMWV 
work,  AM  General  remained  motivated  to  continue  to  compete 
for  the  J  LTV  production  effort. 


Deployment  Phase  Market  Research  (non-EMD  vendors)  para¬ 
graph  in  the  executive  summary:  'Non-EMD  vendors  will  be 
expected  to  perform,  at  a  minimum,  all  of  the  same  testing  in 
the  same  manner  with  the  same  hardware  quantities  as  the 
EMD  vendors.' 


In  the  end,  a  key  contributor  that  motivated  both  Oshkosh  and 
AM  General  to  compete  in  the  EMD-phase  competition  was 
the  major  change  in  user  requirements  driven  by  the  Senate 
subcommittee's  threat  of  program  cancellation.  The  new  user 
requirements  changed  the  overall  technical  focus  of  the  pro¬ 
gram  from  an  expensive,  high-risk  approach  to  an  approach 
with  less  technical  risk  and  a  significantly  lower  cost  in  pro¬ 
duction.  This  change  reduced  the  relevance  of  the  competitive 
prototyping  strategy  employed  in  the  TD  phase,  which  was 
based  on  much  different  JLTV  user  requirements. 

The  JLTV  program  is  currently  in  the  EMD  phase.  The  source 
selection  strategy  for  the  program's  next  phase,  Production 
and  Deployment  (P&D),  has  not  been  finalized  at  this  time. 
Will  the  JLTV  program  simply  choose  to  employ  a  single  in¬ 
dustry  partner  based  on  the  results  of  the  EMD-phase  efforts, 
or  will  competition  continue? 

It  appears  that  "ensuring  competition  throughout  the  pro¬ 
gram  life  cycle''  remains  a  focus  of  the  JLTV  program  office. 
On  March  13,  2013,  the  JLTV  office  released  a  P&D-phase 


The  conditions  for  continued  competition  into  the  next  phase 
appear  to  be  in  place.  Do  these  conditions  represent  a  viable 
opportunity  for  a  non-EMD  vendor  to  win  a  production  con¬ 
tract  for  the  JLTV? 

The  JLTV  program's  acquisition  strategy  includes  a  com¬ 
petitive  focus  well  into  the  P&D  phase  through  the  optional 
purchase  of  the  JLTV  Technical  Data  Package  (TDP).  Owner¬ 
ship  of  the  JLTV  TDP  provides  the  capability  of  competing 
for  follow-on  production  efforts  with  other  industry  partners. 
This  competitive  component  of  the  JLTV  program  was  briefly 
mentioned  in  the  JLTV  Selective  Acquisition  Report  Executive 
Summary  dated  Dec.  31,  2012. 

In  conclusion,  the  JLTV  program  has  had  significant  competi¬ 
tion  to  date  and  clearly  supports  the  concept  of  maintaining 
competitive  environments.  The  decision  to  continue  compe¬ 
tition  into  the  EMD  phase  with  three  offerors  is  costly,  but  it 
clearly  ensures  the  continuation  of  competition  through  the 
P&D  phase.  The  unusual  occurrence  of  two  of  three  TD-phase 
losers  winning  EMD-phase  contracts  is  an  event  worthy  of 
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market  survey  for  the  following  reason:  "The  purpose  of  this 
market  survey  is  to  determine  whether  there  are  any  viable 
non-EMD  vendors  able  to  compete  on  a  full  and  open  basis  in 
accordance  with  the  acquisition  strategy  of  the  J  LTV  program." 

This  12-page  document  outlines  in-depth  requirements  for 
any  interested  industry  partner  that  desires  to  compete  for  the 
final  production  contract.  It  leaves  the  door  open  for  continued 
competition  into  the  production  phase  of  the  program.  This 
market  survey  also  clearly  states  what  potential,  non-EMD 
vendors  must  do  to  be  competitive  in  the  event  they  submit  a 
bid  for  the  upcoming  JLTV  production  effort: 

The  JLTV  EMD  solicitation  (W56HZV-11-R-0329)  offered  non- 
EMD  vendors  the  opportunity  to  participate  in  the  JLTV  EMD 
program  at  their  own  risk  and  expense.  Per  the  Production  and 


examination.  For  this  to  occur,  those  teams  (Oshkosh  and  AM 
General)  had  to  use  significant  internal  fundingto  be  competi¬ 
tive.  What  was  their  motivation  for  doing  so?  I  believe  part  of 
the  answer  lies  in  the  changes  to  user  requirements,  which 
made  these  teams  more  competitive.  The  recent  release  of 
the  market  survey  and  the  option  to  purchase  the  TDP  for 
possible  competition  for  additional  production  contracts  are 
positive  developments  from  a  competition  perspective.  The 
big  question  is  whether  any  non-EMD  offeror  could  meet  the 
rigorous  requirements  outlined  in  the  market  survey.  A  critical 
question  yet  to  be  answered  is  whether  the  JLTV  acquisition 
strategy,  which  maximizes  competition,  will  result  in  a  fielded 
product  that  meets  user  requirements  and  the  established 
cost  cap  per  vehicle  of  $250,000.  Only  time  will  tell. 


The  author  can  be  contacted  at  steve.mills@dau.mil. 
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Your  PM  Personality 

—and  Why  It  Matters 


Owen  Gadeken,  DSci.,  Eng.  Mgt. 


Personality  type  doesn't  sound  like  a  topic  with  much  relevance  to  program  management. 
Most  of  my  DAU  students  in  the  program  management  (PM)  career  path  don't  think  a 
lot  about  the  impact  of  their  personality  or  management  style  until  they  begin  to  have 
problems  on  the  job.  Then  it  slowly  dawns  on  them  that  their  personality  may  be  part  of 
the  problem  as  well  as  the  way  forward  to  a  solution. 

Effectiveness  in  the  acquisition  workplace  requires  much  more  than  just  knowing  what  to  do.  You  have  to  be  able 
to  apply  your  knowledge  in  a  variety  of  complex  situations  where  "by  the  book"  solutions  often  fail.  This  gives  rise 
to  the  "knowing-doing  gap"  or  the  inability  to  translate  subject-matter  expertise  into  workplace  results. 

As  a  manager,  your  results  depend  much  less  on  what  you  know  and  do,  and  much  more  on  what  your  team  knows 
and  can  do.  Therein  lies  the  link  between  personality  and  results.  Your  personality  impacts  your  management  style, 
and  your  management  style  impacts  how  you  interact  with  your  team  to  achieve  results. 

Every  manager  seeks  to  provide  direction  and  motivation  to  his  or  her  team,  but  management  style  can  often 
get  in  the  way.  Every  team  member  is  different.  Each  one  has  different  experiences,  skill  levels,  motivation  and 
different  ways  (styles)  by  which  they  like  to  be  managed.  So,  effective  managers  take  time  to  analyze  their  style 


Gadeken  teaches  program  management  at  the  Defense  Acquisition  University  at  Fort  Belvoir ;  Va.  He  holds  a  doctorate  of  science  in  engineer¬ 
ing  management  from  George  Washington  University  and  is  a  retired  air  force  officer  and  acquisition  professional. 
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and  the  style  of  each  of  their 
team  members,  and  often 
adapt  their  style  with  differ¬ 
ent  team  members  to  get 
better  results. 

One  tool  that  has  proven 
quite  useful  in  analyzing 
people's  styles  at  work  is  the 
Myers-Briggs  (personality) 
Type  Indicator  (MBTI).  This 
assessment  can  be  done  in 
a  few  minutes  online  or  with 
a  self-score  instrument  in 
class  and  is  quite  easy  to 
understand  when  presented 
by  a  trained  facilitator.  In 
many  of  our  classes,  most 
students  have  already  taken 
the  MBTI  but  have  not  had 
enough  training  or  practice 
in  how  to  use  it. 


Figure  1.  MBTI®  Type  Table  of  PM  Attributes 


ISTJ 

responsible 

structured 

consistent 

ISFJ 

gracious 

dependable 

low-key 

INFJ 

insightful 

caring 

inner-drive 

INTJ 

thoughtful 

independent 

focused 

ISTP 
practical 
immediate 
calm  in  crisis 

ISFP 

easy  going 
supportive 
quiet  action 

INFP 

idealistic 

value-driven 

facilitative 

INTP 

conceptual 

adaptive 

challenger 

ESTP 

action  oriented 
risk  taker 
negotiator 

ESFP 

engaging 

resourceful 

spontaneous 

ENFP 
energetic 
inclusive 
idea  oriented 

ENTP 

freethinker 
theoretical 
natural  debater 

ESTJ 

decisive 

systematic 

accountable 

ESFJ 

debonair 

organized 

traditional 

ENFJ 

enthusiastic 
persuasive 
team  builder 

ENTJ 
strategic 
commanding 
results  focused 

Note:  Myers-Briggs  Type  Indicator  and  MBTI  are  registered  trademarks  of  the  Myers-Briggs  Type  Indicator  Trust 
in  the  United  States  and  other  countries. 


MBTI  is  composed  of  four  basic  personality  preferences 
that  relate  well  to  workplace  activities.  The  first  two  prefer¬ 
ences,  called  the  core  functions,  deal  with  how  we  gather 
information  and  then  make  decisions  based  on  this  infor¬ 
mation.  Sensors  (S)  prefer  to  take  their  information  from 
what  they  see  in  the  present  with  their  five  "senses"— in¬ 
formation  they  often  capture  in  great  detail.  Those  who  are 
iNtuitors  (N)  look  more  at  how  the  information  fits  into  a 
"big  picture"  or  what  future  possibilities  it  suggests.  With 
the  data  gathered,  thinkers  (T)  analyze  it  logically  to  make 
the  most  objective  decision,  while  feelers  (F)  include  the 
impact  on  the  people  involved  to  achieve  a  more  harmoni¬ 
ous  and  subjective  decision. 

The  remaining  two  preferences,  the  first  and  last  letters  in  your 
four-letter  personality  Type,  are  called  the  "attitudes"  since 
they  amplify  the  two  core  functions  discussed  above.  The  first 
letter  of  your  MBTI  Type  has  to  do  with  where  you  get  your 
energy.  Extroverts  (E)  get  their  energy  from  being  in  the  exter¬ 
nal  world  of  people  and  activities,  while  Introverts  (I)  are  more 
comfortable  reflecting  in  their  inner  world  of  thoughts  and 
ideas.  The  fourth  and  last  letter  of  your  MBTI  Type  indicates 
which  of  the  core  functions  you  display  to  the  outside  world. 
If  your  last  letter  is  J,  your  Judging  or  decision-making  func¬ 
tion  is  what  you  show  to  the  world.  You  appear  to  others  as 
structured  and  decisive.  If  on  the  other  hand,  your  last  letter  is 
P,  your  Perceiving  or  data  gathering  function  is  what  you  show 
to  others.  You  appear  open  and  flexible  since  as  a  perceiver 
you  always  are  looking  for  new  information. 

With  MBTI,  it  is  important  to  understand  the  concept  of  pref¬ 
erence.  While  Type  theory  says  we  are  born  with  a  preference 
for  one  letter  in  each  of  these  pairs  (S-N,  T-F,  E-l  &  J-P),  our 
day-to-day  behavior  reflects  a  balancing  act  that  includes  all 


eight  letters.  We  choose  our  behavior,  so  even  the  most  in¬ 
troverted  type  still  must  engage  with  the  outside  world  and 
talk  to  people,  and  the  off-the-wall  extrovert  still  needs  some 
time  alone  to  reflect.  The  key  here  is  to  find  the  right  balance 
between  your  individual  preferences  and  the  demands  created 
by  the  situations  you  face  each  day. 

The  other  key  concept  in  Type  theory  is  self-validation.  The 
results  of  your  MBTI  assessment  are  really  just  a  working  hy¬ 
pothesis,  which  you  must  validate.  In  other  words,  you  decide 
which  of  the  four-letter  Types  (see  Figure  1  Type  Table)  is  the 
best  fit  for  you.  This  is  often  the  Type  from  your  assessment 
report,  but  you  still  must  decide  and  validate  ("take  owner¬ 
ship"  of)  your  Type. 

Many  potential  insights  can  come  from  MBTI,  but  they  all  re¬ 
quire  spending  some  time  reflecting  on  the  Type  you  validate 
and  how  that  Type  has  helped  and  hindered  you  in  your  current 
and  past  assignments.  For  example,  as  an  ENFJ,  I  was  great  at 
putting  together  a  people-centered  vision  as  a  DAU  depart¬ 
ment  chair,  a  role  I  played  in  two  departments  for  more  than 
10  years.  I  wasn't  so  great  at  keeping  track  of  the  details  such 
as  spell  checking  my  slides  and  following  up  on  action  items  I 
had  assigned  to  subordinates.  I  was  very  strong  on  empower¬ 
ing  my  faculty  across  the  board,  until  I  discovered  that  many  of 
them  didn't  want  to  be  empowered  ("just  tell  me  what  to  do"). 
I  was  always  interested  in  talking  with  my  team  to  the  point 
where  my  two  administrative  assistants  actually  asked  me  to 
stop  coming  into  their  office  every  15  minutes  with  a  different 
task  (both  were  introverts).  They  took  our  MBTI  team  -building 
off-site  session  to  heart  and  actually  learned  how  to  manage 
their  boss  (me).  The  point  here  is  that  MBTI  really  helped  me 
and  my  team  work  more  effectively.  Reflecting  on  my  MBTI 
Type  eventually  convinced  me  that  my  best  role  at  DAU  was 
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not  as  a  department  chair  but  back  in  the  classroom  interact¬ 
ing  with  our  customers  (students). 

So,  what  does  MBTI  mean  to  you?  You  actually  can  explore 
that  question  after  you  have  completed  an  MBTI  assessment 
and  reflect  on  your  four-letter  type.  Here  a  few  questions 
which  may  prove  helpful.  How  does  my  type  play  out  in  my 
current  job  environment?  What  type  or  behaviors  do  I  need 
to  demonstrate  on  my  current  program  to  maximize  our  prob¬ 
ability  of  success?  What  are  the  implications  of  my  type  for 
future  career  opportunities?  How  does  my  type  play  out  at 
home  with  my  family  and  friends?  This  list  of  questions  can 
go  on  from  here. 

Let's  take  the  type  we  most  frequently  find  in  our  DAU  PM 
courses,  namely  ISTJ.  This  Type  derives  from  and  is  ideally 
suited  for  the  Department  of  Defense,  military,  program  man¬ 
agement,  and  science  and  engineering  cultures  which  make 
up  defense  acquisition.  But  this  type  also  has  (as  do  all  types) 
some  rather  glaring  weaknesses,  especially  as  you  move  up  in 
the  organization  from  doing  program  management  to  leading  a 
program  or  organization.  Building  on  the  ISTJ  baseline  of  being 
responsible,  structured  and  consistent,  an  ISTJ  PM  also  needs 
the  flexibility  to  behave  at  the  opposite  end  of  each  of  these 
preferences  as  illustrated  in  Figure  2  and  discussed  below. 

While  the  "I"  program  managers  think  and  reflect  carefully 
(often  in  the  comfort  of  their  closed-door  offices)  before  tak¬ 
ing  action,  they  need  E  behavior  to  be  the  visible  spokesper¬ 
sons  for  their  program  and  get  out  and  "manage  by  walking 
around"  their  office  environments.  While  Ss  take  in  and  sort 


vast  amounts  of  project  data,  Ns  help  us  see  where  that  data 
take  us  in  the  future.  While  Ts  make  clear  and  well-organized 
decisions,  Fs  make  sure  the  people  can  "buy  in"  to  these  deci¬ 
sion  and  are  not  lost  in  the  process.  While  Js  make  sure  the 
program  runs  on  schedule  and  key  deliverable  dates  are  met, 
Ps  stay  open  to  new  and  possibly  better  ways  of  meeting  or 
exceeding  these  requirements. 


Figure  2 .  Type  Development  Challenge 


1  think  and  reflect, 
then  act 

E 

visible  spokesperson 
for  the  team 

S  gather  the  real-world 
data 

N 

look  to  the  future 
(plan  &  vision) 

T  make  fact-based 
decisions 

F 

stand  up  for  the 
people  you  lead 

J  Be  decisive,  reach 
closure 

P 

stay  open  to  new 
ideas 

It  has  become  apparent  to  me  that  the  more  you  can  integrate 
across  each  of  the  preferences,  the  better  program  manager 
and  leader  you  will  be.  Instead  of  someone  who  is  "locked  in" 
to  your  box  on  the  Type  Table,  you  can  be  more  of  a  situational 
leader,  not  with  your  preferences  but  with  the  behaviors  you 
choose  to  employ.  Your  better  self-management  and  adaptive 
behaviors  can  end  up  being  a  gift  to  you,  your  people  and  your 
program.  & 

The  author  may  be  contacted  at  owen.gadeken@dau.mil. 
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Our  Troops  Need  Your 

BRAINPOWER 


Here’s  a  way  to  put  it  to  work 


Join  the  best  minds  in  science  and  technology  on  DoDTechipedia— the  new  internal  wiki  for  the 
U.S.  Department  of  Defense.  Post  ideas,  ask  questions,  make  suggestions,  or  share  information 
with  colleagues  you  can’t  reach  now.  It’s  a  way  to  expand  our  brainpower,  focusing  on  rapidly 
responding  to  the  needs  of  the  warfighter. 


HERE'S  HOW  IT  WORKS 

•  Share  your  knowledge.  Every  contribution  counts.  The  more 
you  contribute,  the  more  the  collective  knowledge  base  ex¬ 
pands.  The  wiki  can  easily  be  edited  by  any  user,  broadening 
your  access  to  the  latest  and  best  research  and  ideas.  DoD¬ 
Techipedia  is  open  to  federal  government  employees  and 
contractors  with  Common  Access  Card  or  DTIC  registration. 

•  Connect  across  walls.  Reach  across  command  chains  and 
departmental  divisions  to  find  other  people  working  on 
ideas  and  solutions  that  interest  you.  Discuss  hot  topics. 

Stay  on  top  of  new  trends.  Read  technical  blogs— or  create 
one  of  your  own.  You  don’t  need  to  know  the  right  people— 
you  can  connect  on  the  wiki. 

•  Collaborate.  The  wars  we  are  fighting  today  require  immedi¬ 
ate  solutions.  The  wiki  is  the  biggest  brainstorming  session 
ever  at  DoD.  Network  with  others  working  in  your  areas  of 
interest.  Present  new  ideas  or  technical  challenges.  Stay 
abreast  of  research  and  development  initiatives,  confer¬ 
ences,  and  symposia.  Collaboration  across  DoD  increases 
our  ability  to  identify  challenges  as  they  emerge  and  deliver 
vigorous  solutions  fast. 


https://www.DoDTechipedia.mil 


START  CONTRIBUTING 
TO  DODTECHIPEDIA  NOW 

If  you  have  CAC  or  DTIC  registration, 
you  already  have  access  to  the  wiki. 
Go  to  https://www.DoDTechipedia.mil 
and  log  in.  Once  on  the  wiki,  visit  the 
tutorials  link  to  learn  how  to  add  or 
edit  information. 


THE  INFORMATION  ASSURANCE 
TECHNOLOGY  ANALYSIS  CENTER 
(IATAC)  MAINTAINS  THE  FOLLOW¬ 
ING  TECHNOLOGY  FOCUS  AREAS: 

Information  Assurance: 

Protection  and  defense  of  information  and  IT 
systems 

Information  Warfare: 

Capabilities  used  to  exploit  information  and 
IT  systems 

Networking  Technology: 

Technologies  that  interconnect  groups  and/ 
or  systems 

IATAC  POC:  Rogelio  Raymond 
703-984-0072  or 
raymond_rogelio@bah.com 
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Predicting  Program  Success— 
Not  Child's  Play 

Debra  E.  Hahn 


henever  I  deal  with  stakeholders,  I  am 
reminded  of  the  game  "Rock,  Paper, 
Scissors"  that  I  played  as  a  child.  If 
you're  not  familiar  with  the  game,  two 
players,  on  the  count  of  three,  put  forth 
a  hand  in  a  symbol  representing  a  rock,  paper  or  scis¬ 
sors.  The  winner  of  the  round  is  determined  by  the  re¬ 
lationship  between  the  two  factors:  Paper  covers  rock, 
so  paper  wins;  rock  breaks  scissors,  so  rock  wins;  or 
scissors  cut  paper,  so  scissors  win.  When  I  was  young, 
the  winner  got  to  punch  the  loser  in  the  arm  or  give 
a  two-finger  wet  slap.  Both  consequences 
were  harmless  but  somewhat  painful. 


One  could  argue  that  there  is  a  forecasting 
aspect  of  the  game  based  on  the  player's  last 
three  to  five  throws— i.e.,  past  performance. 
To  win  continuously,  a  player  has  to  guess 
what  symbol  the  other  player  will  throw  and 
then  throw  the  appropriate  winning  symbol. 
It  is  difficult  to  predict  because  every  throw 
has  a  winner  or  loser  based  solely  on  the  situ¬ 
ation  at  the  time,  and  the  situation  is  dynamic. 


This  is  very  similar  to  the  program  management  and 
acquisition  environment  where  predicting  the  currently  ac¬ 
ceptable  set  of  decision  factors  is  critical  to  program  success. 
When  developing  an  acquisition  strategy  or  determining  a  course 


Hahn  has  41  years  of  financial  and  program  management  experience  as  a  mili¬ 
tary  officer r  DoD  civilian  and  as  the  deputy  director  of  Lockheed- Martin's  Center 
of  Performance  Excellence.  At  Defense  Acquisition  University ;  she  served  as  the 
PMT  401  course  manager  for  7  years  and  currently  is  a  professor  of  financial 
management  and  BCF301  course  manager. 
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of  action,  the  hot  or  driving  influence  or  focus  today  likely  will 
change  in  the  next  12  to  18  months  as  political  and  budget¬ 
ary  considerations  change.  Another  complicating  aspect  is 
that  each  headquarter's  directorates  (functional  areas)  have 
different  perspectives  with  associated  desired  outcomes. 
The  various  multiple  perspectives  can  cloud  the  context  of 
the  issue  or  decision  at  hand.  The  decision  process  becomes 
more  complicated  when  you  consider  the  obviously  inconsis¬ 
tent  objectives  of  the  contractor,  user,  different  Services,  the 
Office  of  the  Secretary  of  Defense,  the  whole  Executive  Branch 


because  of  the  dynamic,  contradictory,  interrelated,  piece¬ 
meal  decision  factors  within  an  environment  with  inconsis¬ 
tent  requirements— and  within  DoD— sometimes  unspoken 
requirements.  Obviously,  there  is  no  one  set  of  approved 
properties  of  wicked  problems;  each  author  has  his  or  her 
own  "bookworthy"  set. 

A  quick  review  of  a  recognized  set  of  wicked  problem  factors 
will  demonstrate  how  the  DoD  acquisition  environment  clearly 
fosters  wicked  problems. 


Mh 


Resolving  a  manufacturing  problem  may 
increase  the  weapon  system's  weight,  resolving 
the  weight  problem  may  increase  the  program 
cost.  Resolving  the  cost  problem  can  often 
decrease  risk  mitigation  actions  which  in  turn 
piay  result  in  a  different  technical  problem. 


and  Congress.  Tryingto  find  a  solution  or  make  a  decision  that 
satisfactorily  appeases  each  of  these  stakeholders  with  their 
conflicting  objectives  can  be  mind-numbing.  Yes,  conflicting! 
The  result  is  that  many  times  the  outcome  is  not  the  optimal 
or  even  the  best  decision  but  the  decision  that  placates  the 
majority  of  the  stakeholders.  In  other  words,  the  most  sellable 
solution/decision  is  made. 

As  an  instructor  who  spent  more  than  11  years  teaching  critical 
thinking  and  decision  making  using  '"real"  program  scenarios/ 
case  studies  in  both  government  and  contractor  program  man¬ 
agement  training  courses,  I've  witnessed  this  phenomenon 
firsthand.  Each  student's  experiences,  functional  perspective 
and  current  assignment  set  the  boundaries  of  an  acceptable 
solution.  The  ensuing  emotional  discussions  center  on  the 
assumptions  each  student  has  made  based  on  unique  back¬ 
grounds,  viewpoints  and  biases. 

Considering  there  are  myriad  stakeholders  and  decision  mak¬ 
ers  within  the  Department  of  Defense  (DoD)  acquisition  pro¬ 
cess,  each  with  unique  boundaries,  the  result  is  innumerable, 
diverse  decision  factors.  Therefore,  I  can  unequivocally  state 
that,  regardless  of  your  perspective  (government  or  contrac¬ 
tor)  determining  the  appropriate  set  of  decision  factors  is 
crucial  to  determining  a  sellable  solution.  Unfortunately,  the 
four-  or  five-step  decision-making  process  typically  taught  is 
not  adequate  to  handle  this  multifaceted  and  overly  complex 
environment,  as  it  assumes  an  agreed  upon  set  of  decision 
factors.  Therefore,  a  more  dynamic  process  is  required. 

The  term  generally  associated  with  a  problem  with  dy¬ 
namic  decision  factors  in  a  multistakeholder  environment 
is  "wicked."  A  wicked  problem  is  almost  impossible  to  solve 


There  are  multiple  stakeholders  with  multiple  unique 
requirements  and  decision  factors.  In  addition,  based  on  their 
perspective,  each  stakeholder  may  envision  a  different  prob¬ 
lem  or  acceptable  solution. 

Every  problem  is  unique.  Similar  problem  may  exist,  but  in 
reality  each  similar  problem  has  a  unique  situation  and  impact¬ 
ing  factors.  For  example,  both  the  Joint  Strike  Fighter  and  the 
MQ-4C  Triton  programs  have  technical  and  cost  issues,  but 
the  factors  driving  those  issues  are  uniquely  different. 

The  problems  are  not  discrete;  rather  the  solution  to  one 
problem  tends  to  cause  another  problem.  Resolving  a 
manufacturing  problem  may  increase  the  weapon  system's 
weight,  resolving  the  weight  problem  may  increase  the  pro¬ 
gram  cost.  Resolving  the  cost  problem  can  often  decrease 
risk  mitigation  actions,  which  in  turn  may  result  in  a  differ¬ 
ent  technical  problem. 

It  is  difficult  to  fully  comprehend  the  problem  without  an  in- 
depth  analysis  of  the  choices,  and  when  each  option  is  ana¬ 
lyzed  the  problem  evolves  or  morphs  into  a  different  problem. 
As  a  team  considers  the  implications  or  circumstances  sur¬ 
rounding  various  choices,  either  the  team  identifies  secondary 
or  tertiary  impacts  that  are  unacceptable  or  the  review  in  itself 
changes  the  situation.  In  either  case,  the  original  problem  or 
the  range  of  acceptable  outcomes  is  redefined. 

There  are  multiple  reasonable  solutions  to  a  wicked  problem. 
There  is  not  one  right  solution.  Multiple  solutions  will  provide 
a  wide  range  of  acceptable  outcomes.  And  unfortunately,  a 
good  decision  does  not  guarantee  a  good  outcome,  because 
of  the  dynamic  nature  of  the  environment. 
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There  are  no  "do  overs."  The  implementation  of  the  decision 
significantly  changes  the  environment/situation,  thereby  al¬ 
tering  the  problem.  Because  the  program  environment  and 
resources  have  been  modified,  previously  considered  alterna¬ 
tives  no  longer  are  viable  and  a  new  wicked  problem  emerges. 

In  summary,  wicked  problems  tend  to  be  one-of-a  kind  situa¬ 
tions  without  a  "right"  solution;  a  number  of  possible,  reason¬ 
able  solutions  exist.  Because  of  the  complexity  of  a  wicked 
problem,  its  resolution  typically  creates  another  problem; 
which  is  typically  wicked.  It  is  this  never-ending  cycle  of  wicked 
problem  after  wicked  problem  that  explains  why  the  DoD  ac¬ 
quisition  process  is  inconsistent  and  why  years  of  acquisition 
reform  changes  have  improved  overall  individual  program 
performance  very  little. 

As  I  said  earlier,  selecting  the  right  decision  factors  is  critical  to 
makingthe  best  decision;  this  is  especially  true  in  the  "wicked" 
DoD  acquisition  process.  As  a  longtime  instructor,  I've  studied 
the  brain  and  how  it  functions.  A  human  brain  cannot  simulta¬ 
neously  process  more  than  four  or  five  decision-factors.  The 
human  brain  cannot  assimilate  and  evaluate  more  than  four 
or  five  interrelated  factors  and  consider  more  than  four  to  five 
possible  consequences.  Yet,  the  DoD  program  manager  (and 
milestone  decision  authority)  often  has  10  to  15  "critical"  fac¬ 
tors  and  multiple  objectives  to  consider  in  making  a  decision. 
I  contend  that  it  isn't  the  acquisition  process  that  needs  to  be 
improved,  but  rather  the  acquisition  environment.  Key  deci¬ 
sion  makers  must  accept  the  fact  that  the  DoD  acquisition 
environment  inherently  generates  wicked  problems.  Solving 


Beware  of  simplistic  solutions  to 
complex  problems.  The  slogan 
should  be:  Define  small  and 
resolve  big. 


wicked  problems  requires  a  multifaceted,  dynamic  decision¬ 
making  process  and  the  typical  DoD  6-month  decision-making 
cycle/process  just  doesn't  "cut  the  mustard."  The  acquisition 
environment  must  be  simplified. 

So  the  million-dollar  question  is:  How  do  you  solve  a  wicked 
problem?  The  answer  is  simple  to  state  and  very  difficult  to 
implement.  First  and  foremost,  accept  that  this  is  a  unique 
problem  within  a  distinctive  environment  with  an  exclusive 
set  of  decision  factors.  What  worked  5  years  ago  in  a  similar 
situation  on  another  program  probably  won't  work  in  this  sit¬ 
uation;  it  is  a  different  problem  in  a  different  environment.  Be¬ 
ware  of  the  Lessons  Learned  trap.  I  strongly  believe  we  should 
learn  from  our  mistakes,  but  be  cognizant  of  the  differences 


between  past  and  present  situations  and  cautiously  apply 
lessons  learned. 

Second,  whenever  possible,  break  the  problem  or  decision 
down  into  more  discrete  pieces.  In  order  to  solve  any  prob¬ 
lem,  the  problem  definition  must  be  clear  and  agreed  upon 
by  all  stakeholders.  Because  wicked  problems  are  pervasive 
and  extremely  difficult  to  delimit,  thus  allowing  for  different 
perceptions,  it  is  easier  to  get  agreement  on  smaller  aspects 
of  the  problem  or  desired  outcomes.  The  "bite-size"  pieces 
then  can  be  prioritized.  There  is  a  risk  that  defining  the  prob¬ 
lem  in  "bite-size"  pieces  makes  it  easy  to  fall  into  the  trap  of 
solving  the  smaller  pieces  and  ignoring  the  dynamic,  interre¬ 
lated  aspects  of  the  problem.  Beware  of  simplistic  solutions 
to  complex  problems.  The  slogan  should  be:  Define  small  and 
resolve  big. 

Third,  focus  on  understanding  the  problem's  interdependen¬ 
cies  and  multidimensional  aspects.  This  action  is  extremely 
difficult  to  effectively  employ,  as  we  often  are  completely 
unaware  of  the  interdependencies  until  we  implement  our 
solution— and  days,  weeks  or  months  later  the  secondary 
or  tertiary  impacts  unexpectedly  reveal  themselves.  It  is  the 
multidimensional  aspect  and  requirements  of  the  acquisition 
environment  that  muddles  the  process.  Multidimensional 
problems  require  multidimensional  solutions,  and  most  of  us 
are  novices  in  multidimensional  thinking. 

Finally,  I  believe  the  easiest  way  to  tame  wicked  problems  is  to 
simplify  the  environment  by  agreeing  on  four  or  five  prioritized 


key  decision  factors.  Unfortunately,  it  isn't  easy  because  not 
only  must  they  be  "agreed  upon"  by  all  stakeholders/deci¬ 
sion  makers,  but  they  must  be  adhered  to  regardless  of  the 
stakeholder's  functional  area  or  level  within  the  DoD,  Executive 
Branch  or  Congress.  Alignment  of  decision  factors  is  critical 
to  improvement  of  overall  program  performance.  The  non- 
selected  decision  factors  favored  by  various  "rice  bowl  con¬ 
stituents"  must  truly  be  set  aside.  In  an  overall  context,  better 
decisions  can  be  made  only  when  the  acquisition  environment 
becomes  less  wicked  and  acquisition  decision  factors  and  re¬ 
quirements  become  less  complex  and  more  definitive.  Until 
that  happens,  program  managers  will  continue  to  play  Rock, 
Paper,  Scissors  when  making  decisions.  & 
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